Dictionary Learning Algorithms for Sparse Representation
Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtainin...
Saved in:
| Published in | Neural computation Vol. 15; no. 2; pp. 349 - 396 |
|---|---|
| Main Authors | , , , , , |
| Format | Journal Article |
| Language | English |
| Published |
One Rogers Street, Cambridge, MA 02142-1209, USA
MIT Press
01.02.2003
|
| Subjects | |
| Online Access | Get full text |
| ISSN | 0899-7667 1530-888X 1530-888X |
| DOI | 10.1162/089976603762552951 |
Cover
| Abstract | Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial “25 words or less”), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations.
Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error). |
|---|---|
| AbstractList | Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error). Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial “25 words or less”), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an over-complete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error). Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial “25 words or less”), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error). Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error). |
| Author | Sejnowski, Terrence J. Murray, Joseph F. Kreutz-Delgado, Kenneth Engan, Kjersti Rao, Bhaskar D. Lee, Te-Won |
| Author_xml | – sequence: 1 givenname: Kenneth surname: Kreutz-Delgado fullname: Kreutz-Delgado, Kenneth – sequence: 2 givenname: Joseph F. surname: Murray fullname: Murray, Joseph F. email: jfmurray@ucsd.edu – sequence: 3 givenname: Bhaskar D. surname: Rao fullname: Rao, Bhaskar D. email: brao@ece.ucsd.edu – sequence: 4 givenname: Kjersti surname: Engan fullname: Engan, Kjersti email: kjersti.engan@tn.his.no organization: Stavanger University College, School of Science and Technology Ullandhaug, N-4091 Stavanger, Norway, kjersti.engan@tn.his.no – sequence: 5 givenname: Te-Won surname: Lee fullname: Lee, Te-Won organization: Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, California 92037, U.S.A., tewon@salk.edu – sequence: 6 givenname: Terrence J. surname: Sejnowski fullname: Sejnowski, Terrence J. organization: Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, California 92037, U.S.A., terry@salk.edu |
| BackLink | http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=14582951$$DView record in Pascal Francis https://www.ncbi.nlm.nih.gov/pubmed/12590811$$D View this record in MEDLINE/PubMed |
| BookMark | eNqNkc1u1DAUhS1URKeFF2CBsoFdyrUdO_YGqWopII2ExI_EznIcZ-oqsYOdFJWnx6NEGn6qipUX_r7jc69P0JEP3iL0HMMZxpy8BiFlzTnQmhPGiGT4EdpgRqEUQnw7Qps9UGaiPkYnKd0AAMfAnqBjTJgEgfEGiUtnJhe8jnfF1urond8V5_0uRDddD6noQiw-jzomW3yyY7TJ-knvhafocaf7ZJ-t5yn6evX2y8X7cvvx3YeL821pmORT2VacNxLXlGNR8xq6llZM0rY1YAgxhFLLaytbohstTWcgVxMNaFJ3QgJu6CmiS-7sR333Q_e9GqMbcl-FQe33oP7dQ7beLNY4N4NtTW4d9cEM2qk_b7y7Vrtwq4isKiCQA16tATF8n22a1OCSsX2vvQ1zUjWFilBWZ_DF7y8dyq0rzsDLFdDJ6L6L2huXDlzFxFqZLJyJIaVou_-bU_wlGbf8T57K9Q-rZ4s6uEndhDn6_I0PC1f3CN6acIuZI4oCoUIoAgTnjJygfrrxvqBfzV7M6g |
| CitedBy_id | crossref_primary_10_1002_sca_21127 crossref_primary_10_1016_j_sigpro_2008_12_005 crossref_primary_10_1016_j_ymssp_2020_107434 crossref_primary_10_1109_TSP_2014_2328981 crossref_primary_10_1016_j_ymssp_2010_07_019 crossref_primary_10_3390_rs11222608 crossref_primary_10_1109_TIP_2015_2409565 crossref_primary_10_1186_1687_6180_2011_58 crossref_primary_10_1109_TMI_2014_2336860 crossref_primary_10_1109_TIT_2016_2517011 crossref_primary_10_1109_TSP_2007_896242 crossref_primary_10_1007_s10955_024_03359_9 crossref_primary_10_1007_s11263_015_0799_8 crossref_primary_10_1109_TIP_2013_2282900 crossref_primary_10_1162_NECO_a_00672 crossref_primary_10_4028_www_scientific_net_AMM_556_562_3658 crossref_primary_10_1016_j_sigpro_2012_09_011 crossref_primary_10_1137_07070156X crossref_primary_10_1139_cgj_2023_0457 crossref_primary_10_1785_0220180302 crossref_primary_10_1016_j_bspc_2013_05_003 crossref_primary_10_1137_15M1030376 crossref_primary_10_1049_iet_spr_2019_0242 crossref_primary_10_36548_jtcsst_2021_3_003 crossref_primary_10_1016_j_neunet_2012_08_013 crossref_primary_10_1177_1475921716664515 crossref_primary_10_1162_neco_a_01325 crossref_primary_10_1007_s00500_015_1959_z crossref_primary_10_1016_j_cor_2012_12_002 crossref_primary_10_1002_etep_2405 crossref_primary_10_3934_ipi_2011_5_815 crossref_primary_10_1109_TBC_2016_2638620 crossref_primary_10_1109_MSP_2013_2253354 crossref_primary_10_1364_OPTICA_506813 crossref_primary_10_1155_2018_1532868 crossref_primary_10_1155_2017_7483639 crossref_primary_10_1109_JPROC_2022_3155904 crossref_primary_10_1007_s11042_019_08424_0 crossref_primary_10_1002_mrm_27521 crossref_primary_10_1002_ima_20231 crossref_primary_10_1007_s13246_024_01507_9 crossref_primary_10_1016_j_measurement_2021_109810 crossref_primary_10_1016_j_specom_2013_01_005 crossref_primary_10_3389_fnimg_2022_963125 crossref_primary_10_1007_s10851_008_0120_3 crossref_primary_10_1137_120863307 crossref_primary_10_1109_TNNLS_2016_2532358 crossref_primary_10_1016_j_sigpro_2016_02_008 crossref_primary_10_1109_TSP_2008_919392 crossref_primary_10_1007_s11682_015_9359_7 crossref_primary_10_1016_j_neucom_2015_05_036 crossref_primary_10_1007_s11265_006_9774_5 crossref_primary_10_1109_TIT_2006_876348 crossref_primary_10_1109_LSP_2015_2454991 crossref_primary_10_1109_TSP_2015_2405503 crossref_primary_10_1002_mp_13337 crossref_primary_10_1109_TBME_2017_2737785 crossref_primary_10_1155_2021_8864981 crossref_primary_10_1186_s13634_018_0538_8 crossref_primary_10_1016_j_jvcir_2012_04_003 crossref_primary_10_1016_j_cam_2016_03_011 crossref_primary_10_1016_j_dsp_2011_10_006 crossref_primary_10_1016_j_bspc_2023_105856 crossref_primary_10_1109_LSP_2014_2345761 crossref_primary_10_1109_MAP_2020_3043469 crossref_primary_10_1007_s00034_021_01675_z crossref_primary_10_1109_TIP_2017_2686014 crossref_primary_10_1109_JSTSP_2011_2161264 crossref_primary_10_1109_TCYB_2016_2612686 crossref_primary_10_1109_TWC_2019_2952843 crossref_primary_10_1109_TWC_2018_2843786 crossref_primary_10_2478_amns_2023_1_00337 crossref_primary_10_1016_j_neucom_2007_08_034 crossref_primary_10_1016_j_ymssp_2018_07_036 crossref_primary_10_1007_s11265_006_0003_z crossref_primary_10_1109_TSP_2021_3100305 crossref_primary_10_1162_neco_a_01304 crossref_primary_10_1109_TASL_2007_909434 crossref_primary_10_1109_TGRS_2016_2594952 crossref_primary_10_1109_TNN_2005_853426 crossref_primary_10_1109_TSP_2015_2472372 crossref_primary_10_1109_TIP_2013_2246175 crossref_primary_10_1109_TGRS_2015_2484619 crossref_primary_10_1002_mrm_21202 crossref_primary_10_1016_j_sigpro_2005_06_007 crossref_primary_10_1007_s11517_015_1382_8 crossref_primary_10_1088_0031_9155_58_16_5803 crossref_primary_10_3390_electronics14020353 crossref_primary_10_3390_s21082602 crossref_primary_10_1016_j_bspc_2017_08_007 crossref_primary_10_1016_j_oceaneng_2021_109013 crossref_primary_10_1177_1687814017703008 crossref_primary_10_1162_neco_2009_08_08_846 crossref_primary_10_1016_j_imavis_2013_12_015 crossref_primary_10_1109_TSP_2012_2218807 crossref_primary_10_1109_ACCESS_2020_3007150 crossref_primary_10_1137_19M126044X crossref_primary_10_1016_j_patcog_2017_11_002 crossref_primary_10_1109_ACCESS_2019_2930236 crossref_primary_10_1109_TCSVT_2016_2539778 crossref_primary_10_1109_TMI_2015_2418734 crossref_primary_10_1142_S021945542340028X crossref_primary_10_1109_TNN_2010_2049370 crossref_primary_10_1016_j_imavis_2014_02_007 crossref_primary_10_1109_TIT_2016_2556702 crossref_primary_10_1038_srep37862 crossref_primary_10_1137_090770783 crossref_primary_10_1016_j_neucom_2011_02_002 crossref_primary_10_1109_ACCESS_2019_2932098 crossref_primary_10_1089_neur_2023_0059 crossref_primary_10_1007_s13042_018_0856_z crossref_primary_10_1007_s10851_013_0431_x crossref_primary_10_1049_iet_spr_2014_0072 crossref_primary_10_1109_TCSVT_2014_2317886 crossref_primary_10_3390_jimaging4050068 crossref_primary_10_1007_s11460_009_0009_y crossref_primary_10_1109_TCBB_2023_3321593 crossref_primary_10_1186_s13634_018_0533_0 crossref_primary_10_1016_j_jappgeo_2024_105475 crossref_primary_10_1109_JPROC_2009_2030345 crossref_primary_10_1088_1742_6596_1237_3_032021 crossref_primary_10_1016_j_neucom_2014_03_034 crossref_primary_10_1109_TSP_2020_3012946 crossref_primary_10_1109_JSEN_2023_3325364 crossref_primary_10_1109_TNSRE_2014_2319334 crossref_primary_10_1016_j_acha_2014_01_005 crossref_primary_10_1109_JSTSP_2010_2042410 crossref_primary_10_1016_j_sigpro_2025_109886 crossref_primary_10_1109_TIP_2007_901813 crossref_primary_10_1016_j_cageo_2016_03_009 crossref_primary_10_1121_1_2982414 crossref_primary_10_1016_j_optcom_2012_10_014 crossref_primary_10_1109_ACCESS_2019_2954693 crossref_primary_10_1016_j_compgeo_2023_105328 crossref_primary_10_1109_TIP_2013_2290871 crossref_primary_10_1109_TSP_2011_2157146 crossref_primary_10_1007_s11042_021_10516_9 crossref_primary_10_1109_TBME_2016_2533722 crossref_primary_10_1109_TNNLS_2014_2323985 crossref_primary_10_1137_060657704 crossref_primary_10_3390_bioengineering10040397 crossref_primary_10_1049_iet_cvi_2016_0186 crossref_primary_10_1364_AO_51_003941 crossref_primary_10_1162_089976604773717586 crossref_primary_10_1007_s41365_022_01014_0 crossref_primary_10_1088_1742_6596_1592_1_012066 crossref_primary_10_1109_TSP_2009_2026610 crossref_primary_10_1109_TIT_2010_2048466 crossref_primary_10_3390_app13095749 crossref_primary_10_1007_s11760_013_0429_2 crossref_primary_10_1016_j_ress_2018_02_010 crossref_primary_10_1016_j_neucom_2011_06_033 crossref_primary_10_1109_TIT_2011_2165821 crossref_primary_10_1155_2019_3280961 crossref_primary_10_1080_10589759_2011_577428 crossref_primary_10_1121_1_5133944 crossref_primary_10_1002_mp_13415 crossref_primary_10_1049_iet_ipr_2012_0554 crossref_primary_10_1007_s11042_018_6090_6 crossref_primary_10_1016_j_sigpro_2013_11_032 crossref_primary_10_1016_j_ndteint_2007_05_002 crossref_primary_10_1109_JSTARS_2017_2754553 crossref_primary_10_1016_j_dsp_2009_02_001 crossref_primary_10_1109_TIP_2006_881969 crossref_primary_10_1007_s10586_017_0823_6 crossref_primary_10_1109_TBME_2018_2806958 crossref_primary_10_1146_annurev_neuro_062111_150410 crossref_primary_10_1016_j_jmapro_2023_08_040 crossref_primary_10_1063_5_0202931 crossref_primary_10_1121_1_2215407 crossref_primary_10_1109_TUFFC_2015_007048 crossref_primary_10_1109_TVLSI_2014_2301733 crossref_primary_10_1145_3141770 crossref_primary_10_1007_s11042_022_13444_4 crossref_primary_10_1016_j_ijhydene_2023_03_373 crossref_primary_10_1109_TSP_2008_2007605 crossref_primary_10_1162_NECO_a_00907 crossref_primary_10_1109_TSP_2020_2971948 crossref_primary_10_1117_1_OE_57_7_073102 crossref_primary_10_1109_TMI_2010_2097275 crossref_primary_10_1016_j_cam_2016_09_019 crossref_primary_10_1016_j_sigpro_2018_10_010 crossref_primary_10_3390_math11122674 crossref_primary_10_1109_TSP_2009_2016257 crossref_primary_10_1007_s10712_024_09828_w crossref_primary_10_1016_j_eswa_2013_11_036 crossref_primary_10_1109_TIP_2017_2681436 crossref_primary_10_1137_18M121976X crossref_primary_10_3390_jmse9111245 crossref_primary_10_1109_TSP_2008_928160 crossref_primary_10_1007_s11042_016_4134_3 crossref_primary_10_1038_s41598_017_18776_y crossref_primary_10_1109_TBME_2013_2262099 crossref_primary_10_1016_j_sigpro_2009_04_001 crossref_primary_10_1137_16M1091447 crossref_primary_10_1109_TSIPN_2017_2710905 crossref_primary_10_1109_TWC_2019_2915955 crossref_primary_10_1016_j_compbiomed_2016_01_013 crossref_primary_10_1109_TCI_2017_2663321 crossref_primary_10_1109_ACCESS_2019_2959011 crossref_primary_10_1007_s11432_014_5258_6 crossref_primary_10_1016_j_jmr_2015_11_003 crossref_primary_10_1049_rsn2_12348 crossref_primary_10_1103_PhysRevE_106_024136 crossref_primary_10_1109_TSP_2010_2040671 crossref_primary_10_3390_e20030150 crossref_primary_10_1007_s11440_024_02475_w crossref_primary_10_1109_TSP_2006_873367 crossref_primary_10_1007_s11042_020_09850_1 crossref_primary_10_1016_j_acha_2013_10_001 crossref_primary_10_1109_TIP_2010_2081679 crossref_primary_10_1177_20552076241302234 crossref_primary_10_1016_j_ultras_2011_10_001 crossref_primary_10_1109_TSP_2010_2044251 crossref_primary_10_3390_bdcc6020044 crossref_primary_10_1109_TPAMI_2013_236 crossref_primary_10_1177_10775463231212528 crossref_primary_10_1523_JNEUROSCI_1563_06_2006 crossref_primary_10_1002_mrc_4532 crossref_primary_10_1109_JSTSP_2009_2039171 crossref_primary_10_1016_j_jneumeth_2024_110323 crossref_primary_10_1016_j_neucom_2012_10_024 crossref_primary_10_1109_TNNLS_2013_2238682 crossref_primary_10_1109_ACCESS_2015_2430359 crossref_primary_10_1029_2023JB027016 crossref_primary_10_3934_ipi_2012_6_447 crossref_primary_10_1016_j_wavemoti_2024_103375 crossref_primary_10_1109_TNN_2007_891681 crossref_primary_10_3934_ipi_2016012 crossref_primary_10_1088_1751_8121_ad2299 crossref_primary_10_1103_PhysRevE_107_064308 crossref_primary_10_1016_j_measurement_2017_02_031 crossref_primary_10_1364_BOE_10_001660 crossref_primary_10_1016_j_dsp_2018_09_007 crossref_primary_10_1016_j_laa_2005_06_035 crossref_primary_10_1016_j_rinam_2024_100516 crossref_primary_10_1007_s10589_017_9913_x crossref_primary_10_1016_j_acha_2016_08_002 crossref_primary_10_1016_j_conb_2010_07_004 crossref_primary_10_1109_TSP_2018_2807422 crossref_primary_10_1111_j_1467_8659_2009_01495_x crossref_primary_10_1016_j_sigpro_2005_08_009 crossref_primary_10_1002_mp_12097 crossref_primary_10_1093_imaiai_iaad008 crossref_primary_10_1051_matecconf_20165602008 crossref_primary_10_1109_TPAMI_2014_2353631 crossref_primary_10_5802_crphys_236 crossref_primary_10_1109_TCYB_2013_2278548 crossref_primary_10_3390_info13030111 crossref_primary_10_1016_j_ins_2014_08_066 crossref_primary_10_1016_j_image_2019_04_006 crossref_primary_10_1016_j_sigpro_2017_01_004 crossref_primary_10_1190_geo2013_0252_1 crossref_primary_10_2200_S00640ED1V01Y201504IVM018 crossref_primary_10_1109_TPAMI_2012_215 crossref_primary_10_1017_jfm_2016_616 crossref_primary_10_1109_TSP_2013_2277834 crossref_primary_10_1016_j_neuroimage_2015_05_088 crossref_primary_10_1109_TIP_2009_2022459 crossref_primary_10_1088_1742_6596_1982_1_012162 crossref_primary_10_3390_s18103513 crossref_primary_10_1109_TSP_2019_2952046 crossref_primary_10_1016_j_patcog_2011_05_002 crossref_primary_10_1109_ACCESS_2019_2933550 crossref_primary_10_7498_aps_63_189501 crossref_primary_10_1016_j_neucom_2023_126895 crossref_primary_10_1109_TSP_2012_2183129 crossref_primary_10_1007_s00521_014_1607_z crossref_primary_10_1016_j_ultramic_2021_113451 crossref_primary_10_1587_transcom_E96_B_685 crossref_primary_10_1016_j_patcog_2021_107815 crossref_primary_10_1371_journal_pone_0262219 crossref_primary_10_1007_s00041_020_09738_6 crossref_primary_10_1007_s11517_016_1556_z crossref_primary_10_1016_j_neucom_2008_11_027 crossref_primary_10_1109_ACCESS_2019_2944484 crossref_primary_10_1109_TSP_2007_895998 crossref_primary_10_1109_COMST_2017_2664421 crossref_primary_10_1162_neco_2007_19_9_2301 crossref_primary_10_3724_SP_J_1004_2008_00142 crossref_primary_10_1109_MSP_2010_939537 crossref_primary_10_1109_TIT_2011_2162174 crossref_primary_10_3233_ICA_190603 crossref_primary_10_1145_2167076_2167077 crossref_primary_10_1109_TSP_2013_2250968 crossref_primary_10_1109_JSEN_2020_3029707 crossref_primary_10_1109_TSP_2005_861743 crossref_primary_10_1109_TSP_2024_3418971 crossref_primary_10_1038_srep22804 crossref_primary_10_1016_j_bspc_2018_08_016 crossref_primary_10_1016_j_jvcir_2011_09_008 crossref_primary_10_1016_j_measurement_2021_109104 crossref_primary_10_1109_TNNLS_2022_3153310 crossref_primary_10_1016_j_patrec_2010_12_009 crossref_primary_10_1109_TIT_2018_2799931 crossref_primary_10_1109_TIP_2009_2018575 crossref_primary_10_1137_20M1326246 crossref_primary_10_1109_TSP_2012_2215026 crossref_primary_10_1371_journal_pone_0169663 crossref_primary_10_1002_mrm_21757 crossref_primary_10_1007_s10618_016_0461_2 crossref_primary_10_1118_1_4754304 crossref_primary_10_1186_s12888_022_04509_7 crossref_primary_10_1117_1_JRS_10_042003 crossref_primary_10_1016_j_advengsoft_2018_12_004 crossref_primary_10_1016_j_geoen_2024_213230 crossref_primary_10_1088_1742_5468_ac7e4c crossref_primary_10_1016_j_sigpro_2018_01_001 crossref_primary_10_1371_journal_pone_0184667 crossref_primary_10_1049_ip_rsn_20050147 crossref_primary_10_1109_TSP_2015_2486743 crossref_primary_10_1016_j_icarus_2018_09_005 crossref_primary_10_1109_ACCESS_2022_3218032 crossref_primary_10_1007_s10334_022_01029_z crossref_primary_10_1016_j_sigpro_2016_08_015 crossref_primary_10_1109_JSTSP_2011_2158063 crossref_primary_10_1109_JSTSP_2011_2157892 crossref_primary_10_1109_JPROC_2015_2461601 crossref_primary_10_1007_s13246_014_0276_7 crossref_primary_10_1016_j_ijleo_2014_01_003 crossref_primary_10_1109_TSP_2006_881199 crossref_primary_10_1007_s11227_015_1386_z crossref_primary_10_1186_s40068_015_0052_z crossref_primary_10_1109_ACCESS_2018_2880967 crossref_primary_10_1109_TIP_2016_2623484 crossref_primary_10_1115_1_4037419 crossref_primary_10_1107_S1600576721000194 crossref_primary_10_1109_TSA_2005_860346 crossref_primary_10_3390_aerospace8050134 crossref_primary_10_1073_pnas_2314697121 crossref_primary_10_1162_NECO_a_00763 crossref_primary_10_1109_TSP_2013_2278158 crossref_primary_10_1109_TSP_2016_2634546 crossref_primary_10_1016_j_neucom_2018_09_090 crossref_primary_10_1007_s11042_018_7071_5 crossref_primary_10_1016_j_csl_2008_06_002 crossref_primary_10_1038_s41598_017_18860_3 crossref_primary_10_1145_2750780 crossref_primary_10_1016_j_bspc_2015_03_002 crossref_primary_10_1063_5_0135245 crossref_primary_10_1109_TIM_2019_2905043 crossref_primary_10_1088_1361_6501_ac407a crossref_primary_10_1117_1_JEI_25_6_063013 crossref_primary_10_1080_09298210903171178 crossref_primary_10_1109_TASLP_2023_3265199 crossref_primary_10_1186_s43593_024_00078_2 crossref_primary_10_1016_j_patcog_2012_10_010 crossref_primary_10_1121_1_2126935 crossref_primary_10_1631_jzus_2007_A0620 crossref_primary_10_1016_j_cviu_2008_09_003 crossref_primary_10_1080_17499518_2023_2278136 crossref_primary_10_1016_j_image_2016_09_008 crossref_primary_10_1049_iet_spr_2012_0365 crossref_primary_10_1155_2015_560171 crossref_primary_10_1109_TMI_2013_2266259 crossref_primary_10_1109_TNN_2008_2003980 crossref_primary_10_1016_j_compgeo_2023_105953 crossref_primary_10_1016_j_ultras_2006_07_005 crossref_primary_10_1016_j_promfg_2021_06_071 crossref_primary_10_3847_1538_4357_abf48f crossref_primary_10_1016_j_eswa_2021_116192 crossref_primary_10_1109_TMI_2017_2699225 crossref_primary_10_3390_s17020233 crossref_primary_10_1016_j_engappai_2016_08_015 crossref_primary_10_3390_app8122569 crossref_primary_10_1016_j_compag_2017_11_013 crossref_primary_10_1109_TPAMI_2017_2723882 crossref_primary_10_1016_j_dsp_2013_01_004 crossref_primary_10_1016_j_probengmech_2020_103082 crossref_primary_10_1007_s10462_024_10934_9 crossref_primary_10_1142_S0129065720500409 crossref_primary_10_1109_TGRS_2014_2325067 crossref_primary_10_1016_j_measurement_2020_107798 crossref_primary_10_1109_TSP_2012_2187642 crossref_primary_10_1088_0031_9155_52_11_018 crossref_primary_10_1162_neco_2008_03_07_486 crossref_primary_10_1002_adfm_201906041 crossref_primary_10_1109_TIT_2015_2424238 crossref_primary_10_1016_j_advwatres_2011_09_002 crossref_primary_10_1109_JPROC_2010_2040551 crossref_primary_10_1049_iet_cvi_2014_0018 |
| Cites_doi | 10.1038/381607a0 10.1049/ip-vis:19981928 10.1162/neco.1994.6.4.559 10.1109/78.558475 10.1109/78.258082 10.1016/0013-4694(95)00107-A 10.1016/0165-1684(94)90029-9 10.1162/neco.1997.9.8.1627 10.1109/78.542183 10.1162/neco.1995.7.6.1129 10.1162/089976600300015826 10.1016/S0042-6989(00)00290-X 10.1109/78.738251 10.1162/089976699300016458 10.1109/18.119732 10.1016/S0042-6989(97)00169-7 10.1109/18.382009 10.1137/0914086 10.1162/089976601753196003 10.1162/089976601300014385 10.1109/97.551687 10.1088/0954-898X/5/4/006 10.1016/0031-3203(81)90094-7 10.1162/089976699300016719 10.1364/JOSAA.16.001587 10.1088/0954-898X/5/4/008 |
| ContentType | Journal Article |
| Copyright | 2003 INIST-CNRS |
| Copyright_xml | – notice: 2003 INIST-CNRS |
| DBID | AAYXX CITATION IQODW CGR CUY CVF ECM EIF NPM 7X8 5PM ADTOC UNPAY |
| DOI | 10.1162/089976603762552951 |
| DatabaseName | CrossRef Pascal-Francis Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic PubMed Central (Full Participant titles) Unpaywall for CDI: Periodical Content Unpaywall |
| DatabaseTitle | CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic MEDLINE CrossRef |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: EIF name: MEDLINE url: https://proxy.k.utb.cz/login?url=https://www.webofscience.com/wos/medline/basic-search sourceTypes: Index Database – sequence: 3 dbid: UNPAY name: Unpaywall url: https://proxy.k.utb.cz/login?url=https://unpaywall.org/ sourceTypes: Open Access Repository |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science Mathematics Applied Sciences Statistics |
| EISSN | 1530-888X |
| EndPage | 396 |
| ExternalDocumentID | oai:pubmedcentral.nih.gov:2944020 PMC2944020 12590811 14582951 10_1162_089976603762552951 089976603762552951.pdf |
| Genre | Research Support, U.S. Gov't, Non-P.H.S Research Support, Non-U.S. Gov't Journal Article |
| GrantInformation_xml | – fundername: Howard Hughes Medical Institute |
| GroupedDBID | --- -~X .4S .DC 0R~ 123 36B 4.4 6IK AAJGR AALMD ABDBF ABDNZ ABEFU ABIVO ABJNI ABMYL ACGFO ADIYS AEGXH AENEX AFHIN AIAGR ALMA_UNASSIGNED_HOLDINGS ARCSS AVWKF AZFZN BEFXN BFFAM BGNUA BKEBE BPEOZ CAG COF CS3 DU5 EAP EAS EBC EBD EBS ECS EDO EJD EMB EMK EMOBN EPL EPS EST ESX F5P FEDTE FNEHJ HVGLF HZ~ H~9 I-F IPLJI JAVBF MCG MINIK MKJ O9- OCL P2P PK0 PQQKQ RMI SV3 TUS WG8 WH7 XJE ZWS AAYXX ABAZT ABVLG ACUHS ADMLS AMVHM CITATION 41~ 53G AAFWJ ACYGS AEILP IQODW AAYOK CGR CUY CVF ECM EIF NPM 7X8 5PM ABUFD ADTOC UNPAY |
| ID | FETCH-LOGICAL-c596t-d466b91736187670fd34593ddc0c22c233e67e9d2aba9cfc01258b0a27f8901b3 |
| IEDL.DBID | UNPAY |
| ISSN | 0899-7667 1530-888X |
| IngestDate | Sun Oct 26 04:00:25 EDT 2025 Tue Sep 30 16:50:43 EDT 2025 Thu Oct 02 10:08:04 EDT 2025 Fri May 30 10:50:32 EDT 2025 Mon Jul 21 09:17:02 EDT 2025 Wed Oct 01 02:03:00 EDT 2025 Thu Apr 24 23:05:22 EDT 2025 Mon Mar 11 05:40:57 EDT 2024 Tue Mar 01 17:17:49 EST 2022 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 2 |
| Keywords | Stochastic model Compression Dictionaries Description Word Noise Complete Elements Generalization Image Mean square error Learning Probability density Accuracy Schur concavity Gaussian noise Noise source Coding Efficiency Test Bayesian model A posteriori estimation Sparse representation Performance analysis Component Learning algorithm Vector Concept Dictionary learning algorithm Vector quantization Use Independent component analysis Data Representation Neural network Algorithm Neurophysiology Domains Analysis method Algorithm performance Sparse set Environment Density function Models Maximum likelihood Pixel Signal to noise ratio |
| Language | English |
| License | CC BY 4.0 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c596t-d466b91736187670fd34593ddc0c22c233e67e9d2aba9cfc01258b0a27f8901b3 |
| Notes | February, 2003 ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| OpenAccessLink | https://proxy.k.utb.cz/login?url=http://doi.org/10.1162/089976603762552951 |
| PMID | 12590811 |
| PQID | 73042357 |
| PQPubID | 23479 |
| PageCount | 48 |
| ParticipantIDs | unpaywall_primary_10_1162_089976603762552951 mit_journals_10_1162_089976603762552951 proquest_miscellaneous_73042357 pubmed_primary_12590811 crossref_citationtrail_10_1162_089976603762552951 mit_journals_necov15i2_302388_2021_11_08_zip_089976603762552951 pubmedcentral_primary_oai_pubmedcentral_nih_gov_2944020 crossref_primary_10_1162_089976603762552951 pascalfrancis_primary_14582951 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2003-02-01 |
| PublicationDateYYYYMMDD | 2003-02-01 |
| PublicationDate_xml | – month: 02 year: 2003 text: 2003-02-01 day: 01 |
| PublicationDecade | 2000 |
| PublicationPlace | One Rogers Street, Cambridge, MA 02142-1209, USA |
| PublicationPlace_xml | – name: One Rogers Street, Cambridge, MA 02142-1209, USA – name: Cambridge, MA – name: United States |
| PublicationTitle | Neural computation |
| PublicationTitleAlternate | Neural Comput |
| PublicationYear | 2003 |
| Publisher | MIT Press |
| Publisher_xml | – name: MIT Press |
| References | p_41 p_42 p_40 p_16 p_38 p_17 p_39 p_2 p_18 p_19 p_4 p_34 p_35 p_57 p_6 p_58 p_5 p_15 p_53 p_32 p_54 p_11 p_33 p_55 p_50 p_51 |
| References_xml | – ident: p_39 doi: 10.1038/381607a0 – ident: p_51 doi: 10.1049/ip-vis:19981928 – ident: p_15 doi: 10.1162/neco.1994.6.4.559 – ident: p_18 doi: 10.1109/78.558475 – ident: p_35 doi: 10.1109/78.258082 – ident: p_17 doi: 10.1016/0013-4694(95)00107-A – ident: p_6 doi: 10.1016/0165-1684(94)90029-9 – ident: p_57 doi: 10.1162/neco.1997.9.8.1627 – ident: p_41 doi: 10.1109/78.542183 – ident: p_4 doi: 10.1162/neco.1995.7.6.1129 – ident: p_34 doi: 10.1162/089976600300015826 – ident: p_42 doi: 10.1016/S0042-6989(00)00290-X – ident: p_50 doi: 10.1109/78.738251 – ident: p_2 doi: 10.1162/089976699300016458 – ident: p_5 doi: 10.1109/18.119732 – ident: p_40 doi: 10.1016/S0042-6989(97)00169-7 – ident: p_11 doi: 10.1109/18.382009 – ident: p_19 doi: 10.1137/0914086 – ident: p_16 doi: 10.1162/089976601753196003 – ident: p_58 doi: 10.1162/089976601300014385 – ident: p_54 doi: 10.1109/97.551687 – ident: p_53 doi: 10.1088/0954-898X/5/4/006 – ident: p_55 doi: 10.1016/0031-3203(81)90094-7 – ident: p_32 doi: 10.1162/089976699300016719 – ident: p_33 doi: 10.1364/JOSAA.16.001587 – ident: p_38 doi: 10.1088/0954-898X/5/4/008 |
| SSID | ssj0006105 |
| Score | 2.3359642 |
| Snippet | Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary... |
| SourceID | unpaywall pubmedcentral proquest pubmed pascalfrancis crossref mit |
| SourceType | Open Access Repository Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 349 |
| SubjectTerms | Algorithms Applied sciences Artificial Intelligence Biological and medical sciences Computer science; control theory; systems Exact sciences and technology Fundamental and applied biological sciences. Psychology General aspects General aspects. Models. Methods Learning - physiology Learning and adaptive systems Mathematical foundations Mathematics Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects) Probability and statistics Sciences and techniques of general use Statistics Stochastic Processes Vertebrates: nervous system and sense organs |
| Title | Dictionary Learning Algorithms for Sparse Representation |
| URI | https://direct.mit.edu/neco/article/doi/10.1162/089976603762552951 https://www.ncbi.nlm.nih.gov/pubmed/12590811 https://www.proquest.com/docview/73042357 https://pubmed.ncbi.nlm.nih.gov/PMC2944020 http://doi.org/10.1162/089976603762552951 |
| UnpaywallVersion | submittedVersion |
| Volume | 15 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVEBS databaseName: EBSCOhost Academic Search Ultimate customDbUrl: https://search.ebscohost.com/login.aspx?authtype=ip,shib&custid=s3936755&profile=ehost&defaultdb=asn eissn: 1530-888X dateEnd: 20241102 omitProxy: true ssIdentifier: ssj0006105 issn: 0899-7667 databaseCode: ABDBF dateStart: 19970101 isFulltext: true titleUrlDefault: https://search.ebscohost.com/direct.asp?db=asn providerName: EBSCOhost – providerCode: PRVEBS databaseName: Inspec with Full Text customDbUrl: eissn: 1530-888X dateEnd: 20241102 omitProxy: false ssIdentifier: ssj0006105 issn: 0899-7667 databaseCode: ADMLS dateStart: 19970101 isFulltext: true titleUrlDefault: https://www.ebsco.com/products/research-databases/inspec-full-text providerName: EBSCOhost – providerCode: PRVEBS databaseName: Mathematics Source customDbUrl: eissn: 1530-888X dateEnd: 20241102 omitProxy: false ssIdentifier: ssj0006105 issn: 0899-7667 databaseCode: AMVHM dateStart: 19970101 isFulltext: true titleUrlDefault: https://www.ebsco.com/products/research-databases/mathematics-source providerName: EBSCOhost |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fb9MwELagfYAXNn53bCUPSDxAtsRO7PgJVRvThNiEGJXKk-XYzhbRpVWTDm1_PWcnbRe1GvCcuyg5n-3vfOfvEHpnzxUkl9SH9RcCFEq1zxVO_TBLSawV09ox3pye0ZNh9GUUj1Zk0XfT9yHFBzYpxSgNYBoA9sXcXpbu0hhgdwd1h2ffBj8dSuTcBylWc6MGPkR1o8UFmY0vaW1CD6_yypZEyhKsktXtLDbhzfWyyUfzYipvfsvx-M6edLxVV3OVjsrQlqL82p9X6b66XSd6_PvvbqMnDTL1BrUrPUUPTPEMbS26PnjNIvAcJUe5uwshZzdew8564Q3GF5NZXl1elR6gYO98CvGy8b67MtvmdlPxAg2PP_84PPGb_gu-AgtXvoahSyGcIzSENZMFmSZRzInWKlAYK0yIocxwjWUqucoU7HVxkgYSsywBmJGSl6hTTArzGnmR5NhQmsTGMNvUXCqDIxMnNhwyjLAeChcDIlRDTm57ZIyFC1IoFuum6aEPS51pTc1xr_R7GGfRzNDyXslPLcnCqMl1GOdY2AZLSSIw4CJQBlVxm083vqHf8qbV59kspRN4u3AvAdPZ5mhkYSbzUjB7vERiMMir2tlWuti2pw9Bl7XccClgicLbT4r80hGGYx7ZY4Ie-rh02H-w2M7_ib9Bj12ho6to30WdajY3ewDYqrSPuoOj06_n_WbK_gEqtSxu |
| linkProvider | Unpaywall |
| linkToUnpaywall | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fb9MwELage4AXNn53jJEHJB4gW2LHdvw0VcA0ITEhoFJ5shzb2SK6tGrSoe2v39lJ20WtBjznLkrOZ_s73_k7hN66cwUlFAth_YUAhTETCo2zMM4zQo3mxnjGm6-n7GSYfBnR0Yos-nb6Pmb40CWlOGMRTAPAvli4y9JbjALs7qGt4em3wS-PEoUIQYo33KhRCFHdaHFBZuNLOpvQ_YuidiWRqgKr5E07i014c71s8sG8nKqrP2o8vrUnHW831VyVpzJ0pSi_D-Z1dqCv14ke__67O-hRi0yDQeNKj9E9Wz5B24uuD0G7CDxF6afC34VQs6ugZWc9Cwbjs8msqM8vqgBQcPBjCvGyDb77Mtv2dlP5DA2PP__8eBK2_RdCDRauQwNDl0E4R1gMayaPckMSKogxOtIYa0yIZdwKg1WmhM417HU0zSKFeZ4CzMjIc9QrJ6V9iYJECWwZS6m13DU1V9rixNLUhUOWE95H8WJApG7JyV2PjLH0QQrDct00ffR-qTNtqDnulH4H4yzbGVrdKXnUkSytnlzGtMDSNVhKU4kBF4EyqMrrYrrxDfsdb1p9nstSeoE3C_eSMJ1djkaVdjKvJHfHS4SCQV40zrbSxa49fQy6vOOGSwFHFN59UhbnnjAci8QdE_TRh6XD_oPFdv9P_BV66AsdfUX7HurVs7l9DYCtzvbbqXoDz3Iq2g |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Dictionary+learning+algorithms+for+sparse+representation&rft.jtitle=Neural+computation&rft.au=Kreutz-Delgado%2C+Kenneth&rft.au=Murray%2C+Joseph+F&rft.au=Rao%2C+Bhaskar+D&rft.au=Engan%2C+Kjersti&rft.date=2003-02-01&rft.issn=0899-7667&rft.volume=15&rft.issue=2&rft.spage=349&rft_id=info:doi/10.1162%2F089976603762552951&rft_id=info%3Apmid%2F12590811&rft.externalDocID=12590811 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0899-7667&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0899-7667&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0899-7667&client=summon |