Automated Evaluation of School Children Essays in Arabic

The most effective technique for improving students writing skills is for them to get immediate instructor’s feedback and as often as possible. This, however, significantly expands the workload of the instructors. There is a growing need for automated systems to help students draft essays. Automated...

Full description

Saved in:
Bibliographic Details
Published inProcedia computer science Vol. 117; pp. 19 - 22
Main Authors Al-Jouie, Maram F., Azmi, Aqil M.
Format Journal Article
LanguageEnglish
Published Elsevier B.V 2017
Subjects
Automatic essay scoring
Essay evaluation
Latent Semantic Analysis
Rhetorical Structure Theory
Latent Semantic Analysis
Rhetorical Structure Theory
Automatic essay scoring
Essay evaluation
Online AccessGet full text
ISSN1877-0509
1877-0509
DOI10.1016/j.procs.2017.10.089

Cover

Abstract The most effective technique for improving students writing skills is for them to get immediate instructor’s feedback and as often as possible. This, however, significantly expands the workload of the instructors. There is a growing need for automated systems to help students draft essays. Automated essay evaluation is increasingly popular in the field of educational evaluation technology. In this work, we present an automatic evaluator of student essays in Arabic language, a system that is modeled on the scheme followed by the school-teachers in Riyadh, the capital of Saudi Arabia. The main criteria for assessing the essays is: language proficiency; the structure of the essay; and the content which should match the topic. With this in mind, we developed a scheme that relies on latent semantic analysis, and rhetorical structure theory. The system was tested on over 300 different essays, all handwritten by schoolchildren covering various subjects. The performance was measured by machine-human correlation in grading. Our system achieved an overall correlation of 0.79 with the teachers’ evaluation.
AbstractList The most effective technique for improving students writing skills is for them to get immediate instructor’s feedback and as often as possible. This, however, significantly expands the workload of the instructors. There is a growing need for automated systems to help students draft essays. Automated essay evaluation is increasingly popular in the field of educational evaluation technology. In this work, we present an automatic evaluator of student essays in Arabic language, a system that is modeled on the scheme followed by the school-teachers in Riyadh, the capital of Saudi Arabia. The main criteria for assessing the essays is: language proficiency; the structure of the essay; and the content which should match the topic. With this in mind, we developed a scheme that relies on latent semantic analysis, and rhetorical structure theory. The system was tested on over 300 different essays, all handwritten by schoolchildren covering various subjects. The performance was measured by machine-human correlation in grading. Our system achieved an overall correlation of 0.79 with the teachers’ evaluation.
Author Al-Jouie, Maram F.
Azmi, Aqil M.
Author_xml – sequence: 1
  givenname: Maram F.
  surname: Al-Jouie
  fullname: Al-Jouie, Maram F.
  organization: Department of Computer Science, College of Computer & Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
– sequence: 2
  givenname: Aqil M.
  surname: Azmi
  fullname: Azmi, Aqil M.
  email: aqil@ksu.edu.sa
  organization: Department of Computer Science, College of Computer & Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia
BookMark eNqNkM1OAjEQgBuDiYg8gZe-wK7t7mLbgwdCUExIPKjnpjvthpLSkrZgeHsX8GA8qHOZySTf_HzXaOCDNwjdUlJSQu_v1uU2BkhlRSjrOyXh4gINKWesIBMiBt_qKzROaU36qDkXlA0Rn-5y2KhsNJ7vldupbIPHocOvsArB4dnKOh2Nx_OU1CFh6_E0qtbCDbrslEtm_JVH6P1x_jZbFMuXp-fZdFlALWgu6lpXTaN4oya60y3RjaoIcCGAAnBiKjMxLRgGLTDRHwUVIaxlqms7DoJX9Qg157k7v1WHD-Wc3Ea7UfEgKZFHAXItTwLkUcCx2QvoMXHGIIaUoukk2Hx6Lkdl3R9s_YP938aHM2V6G3trokxgjQejbTSQpQ72V_4TSQqLsQ
CitedBy_id crossref_primary_10_1016_j_ipm_2019_05_008
crossref_primary_10_47836_pjst_29_3_27
crossref_primary_10_1109_ACCESS_2024_3470728
crossref_primary_10_1109_ACCESS_2021_3074950
Cites_doi 10.1109/CGC.2012.41
10.2190/G649-0R9C-C021-P6X3
10.3233/AIC-130586
10.1109/APCCAS.2010.5775058
10.1076/1049-4820(200008)8:2;1-B;FT149
10.1109/AMS.2007.19
ContentType Journal Article
Copyright 2017
Copyright_xml – notice: 2017
DBID 6I.
AAFTH
AAYXX
CITATION
ADTOC
UNPAY
DOI 10.1016/j.procs.2017.10.089
DatabaseName ScienceDirect Open Access Titles
Elsevier:ScienceDirect:Open Access
CrossRef
Unpaywall for CDI: Periodical Content
Unpaywall
DatabaseTitle CrossRef
DatabaseTitleList
Database_xml – sequence: 1
  dbid: UNPAY
  name: Unpaywall
  url: https://proxy.k.utb.cz/login?url=https://unpaywall.org/
  sourceTypes: Open Access Repository
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1877-0509
EndPage 22
ExternalDocumentID 10.1016/j.procs.2017.10.089
10_1016_j_procs_2017_10_089
S1877050917321464
GroupedDBID --K
0R~
0SF
1B1
457
5VS
6I.
71M
AACTN
AAEDT
AAEDW
AAFTH
AAIKJ
AALRI
AAQFI
AAXUO
ABMAC
ACGFS
ADBBV
ADEZE
AEXQZ
AFTJW
AGHFR
AITUG
ALMA_UNASSIGNED_HOLDINGS
AMRAJ
E3Z
EBS
EJD
EP3
FDB
FNPLU
HZ~
IXB
KQ8
M41
M~E
NCXOZ
O-L
O9-
OK1
P2P
RIG
ROL
SES
SSZ
AAYWO
AAYXX
ABWVN
ACRPL
ACVFH
ADCNI
ADNMO
ADVLN
AEUPX
AFPUW
AIGII
AKBMS
AKRWK
AKYEP
CITATION
~HD
ADTOC
UNPAY
ID FETCH-LOGICAL-c391t-33d244a84a5dfdb0d4a20c899c1cc80e2e5ebce7cbc79388c2007b7afbf8c9823
IEDL.DBID IXB
ISSN 1877-0509
IngestDate Tue Aug 19 16:57:26 EDT 2025
Thu Apr 24 23:05:34 EDT 2025
Wed Oct 01 03:32:06 EDT 2025
Tue May 16 23:59:33 EDT 2023
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Keywords Latent Semantic Analysis
Rhetorical Structure Theory
Automatic essay scoring
Essay evaluation
Language English
License This is an open access article under the CC BY-NC-ND license.
cc-by-nc-nd
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c391t-33d244a84a5dfdb0d4a20c899c1cc80e2e5ebce7cbc79388c2007b7afbf8c9823
OpenAccessLink https://www.sciencedirect.com/science/article/pii/S1877050917321464
PageCount 4
ParticipantIDs unpaywall_primary_10_1016_j_procs_2017_10_089
crossref_citationtrail_10_1016_j_procs_2017_10_089
crossref_primary_10_1016_j_procs_2017_10_089
elsevier_sciencedirect_doi_10_1016_j_procs_2017_10_089
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2017
2017-00-00
PublicationDateYYYYMMDD 2017-01-01
PublicationDate_xml – year: 2017
  text: 2017
PublicationDecade 2010
PublicationTitle Procedia computer science
PublicationYear 2017
Publisher Elsevier B.V
Publisher_xml – name: Elsevier B.V
References Chen, H., and B. He. (2012). “A Ranked-based Learning Approach To Automated Essay Scoring,”
pp. 448-455.
Page (bib0007) 1968; 14
Wiemer-Hastings, Graesser (bib0009) 2000; 8
Page (bib0006) 1966; 47
.
Kuala Lumpur, Malaysia, pp. 1167-1170.
Nahar, Alsmadi (bib0005) 2009; 1
no. 21. Available at
Alghamdi, Alkanhal, Al-Badrashiny, Al-Qabbany, Areshey, Alharbi (bib0001) 2014; 27
Zhang. M. (2013). “Contrasting Automated and Human Scoring of Essays,”
Lemaire, Dessus (bib0003) 2001; 24
Loraksa, Peachavanish (bib0004) 2007
Razon, A.R., M.L.J. Vargas, R.C.L. Guevara, and P. C. Naval. (2010). “Automated Essay Content Analysis based on Concept Indexing with Fuzzy C-means Clustering,”
Alghamdi (10.1016/j.procs.2017.10.089_bib0001) 2014; 27
Lemaire (10.1016/j.procs.2017.10.089_bib0003) 2001; 24
10.1016/j.procs.2017.10.089_bib00010
Page (10.1016/j.procs.2017.10.089_bib0006) 1966; 47
Loraksa (10.1016/j.procs.2017.10.089_bib0004) 2007
10.1016/j.procs.2017.10.089_bib0008
Page (10.1016/j.procs.2017.10.089_bib0007) 1968; 14
Wiemer-Hastings (10.1016/j.procs.2017.10.089_bib0009) 2000; 8
Nahar (10.1016/j.procs.2017.10.089_bib0005) 2009; 1
10.1016/j.procs.2017.10.089_bib0002
References_xml – reference: Zhang. M. (2013). “Contrasting Automated and Human Scoring of Essays,”
– volume: 14
  start-page: 253
  year: 1968
  end-page: 263
  ident: bib0007
  article-title: “The Use of the Computer in Analyzing Student Essays,”
  publication-title: International Review of Education
– volume: 27
  start-page: 103
  year: 2014
  end-page: 111
  ident: bib0001
  article-title: “A hybrid automatic scoring system for Arabic essays,”
  publication-title: AI Communications
– reference: Razon, A.R., M.L.J. Vargas, R.C.L. Guevara, and P. C. Naval. (2010). “Automated Essay Content Analysis based on Concept Indexing with Fuzzy C-means Clustering,”
– start-page: 400
  year: 2007
  end-page: 402
  ident: bib0004
  article-title: “Automatic Thai-language essay scoring using neural network and latent semantic analysis,”
  publication-title: First Asia International Conference on Modelling Simulation (AMS’07)
– reference: , Kuala Lumpur, Malaysia, pp. 1167-1170.
– volume: 1
  start-page: 215
  year: 2009
  end-page: 220
  ident: bib0005
  article-title: “The automatic grading for online exams in Arabic with essay questions using statistical and computational linguistics techniques,”
  publication-title: MASAUM Journal of Computing
– reference: no. 21. Available at:
– reference: .
– reference: , pp. 448-455.
– volume: 47
  start-page: 238
  year: 1966
  end-page: 243
  ident: bib0006
  article-title: “The imminence of Grading Essays by Computer,”
  publication-title: Phi Delta Kappan
– volume: 8
  start-page: 149
  year: 2000
  end-page: 169
  ident: bib0009
  article-title: “Select-a-Kibitzer: A computer tool that gives meaningful feedback on student compositions.”
  publication-title: Interactive Learning Environments
– reference: Chen, H., and B. He. (2012). “A Ranked-based Learning Approach To Automated Essay Scoring,”
– volume: 24
  start-page: 305
  year: 2001
  end-page: 320
  ident: bib0003
  article-title: “A System to Assess the Semantic Content of Student Essays.”
  publication-title: J. Educational Computing Research
– ident: 10.1016/j.procs.2017.10.089_bib0002
  doi: 10.1109/CGC.2012.41
– ident: 10.1016/j.procs.2017.10.089_bib00010
– volume: 24
  start-page: 305
  year: 2001
  ident: 10.1016/j.procs.2017.10.089_bib0003
  article-title: “A System to Assess the Semantic Content of Student Essays.”
  publication-title: J. Educational Computing Research
  doi: 10.2190/G649-0R9C-C021-P6X3
– volume: 27
  start-page: 103
  issue: 2
  year: 2014
  ident: 10.1016/j.procs.2017.10.089_bib0001
  article-title: “A hybrid automatic scoring system for Arabic essays,”
  publication-title: AI Communications
  doi: 10.3233/AIC-130586
– volume: 1
  start-page: 215
  issue: 2
  year: 2009
  ident: 10.1016/j.procs.2017.10.089_bib0005
  article-title: “The automatic grading for online exams in Arabic with essay questions using statistical and computational linguistics techniques,”
  publication-title: MASAUM Journal of Computing
– ident: 10.1016/j.procs.2017.10.089_bib0008
  doi: 10.1109/APCCAS.2010.5775058
– volume: 8
  start-page: 149
  year: 2000
  ident: 10.1016/j.procs.2017.10.089_bib0009
  article-title: “Select-a-Kibitzer: A computer tool that gives meaningful feedback on student compositions.”
  publication-title: Interactive Learning Environments
  doi: 10.1076/1049-4820(200008)8:2;1-B;FT149
– start-page: 400
  year: 2007
  ident: 10.1016/j.procs.2017.10.089_bib0004
  article-title: “Automatic Thai-language essay scoring using neural network and latent semantic analysis,”
  publication-title: First Asia International Conference on Modelling Simulation (AMS’07)
  doi: 10.1109/AMS.2007.19
– volume: 47
  start-page: 238
  year: 1966
  ident: 10.1016/j.procs.2017.10.089_bib0006
  article-title: “The imminence of Grading Essays by Computer,”
  publication-title: Phi Delta Kappan
– volume: 14
  start-page: 253
  issue: 3
  year: 1968
  ident: 10.1016/j.procs.2017.10.089_bib0007
  article-title: “The Use of the Computer in Analyzing Student Essays,”
  publication-title: International Review of Education
SSID ssj0000388917
Score 2.15106
Snippet The most effective technique for improving students writing skills is for them to get immediate instructor’s feedback and as often as possible. This, however,...
SourceID unpaywall
crossref
elsevier
SourceType Open Access Repository
Enrichment Source
Index Database
Publisher
StartPage 19
SubjectTerms Automatic essay scoring
Essay evaluation
Latent Semantic Analysis
Rhetorical Structure Theory
SummonAdditionalLinks – databaseName: Unpaywall
  dbid: UNPAY
  link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA66HTw5f-JEJQePZqy_0uQ4xsYQHB4czFNJXhOYjm7YFpl_vcmaDhUd81ry2vJe2ve99n3fQ-g25bGIPGCEKWYpOZ4gUjNNwDJnQgkx1Zbg_DCmo0l4P42mTmfbcmG-_b9f92HZF7nV1fbijm3DYnwfNWlkgHcDNSfjx96zLalYHBOrZFLrCv1u-VfuOSizpVi9i_n8S24ZtirSdr6WJLQtJa-dspAd-Pgh2LjjbR-hQ4cxca_aFMdoT2UnqFXPb8DucT5FrFcWC4NYVYoHG9FvvNC4kubEfUf0xoM8F6sczzJzTiFncIYmw8FTf0TcJAUCAfcKEgSpSeOChSJKdSq7aSj8LphSCzwA1lW-ipQEFYMJDg8YA_sFU8ZCm7gBZ35wjhrZIlMXCCtjSKWnQAc6pMrjpr5TlEPqcyrBV23k1z5OwMmM22kX86TuJ3tJ1r5JrG_sQeObNrrbGC0rlY3ty2kdvMQBhQoAJCYO2w3JJtS7XOjyn-uvUKN4K9W1wSmFvHH78xNzQObM
  priority: 102
  providerName: Unpaywall
Title Automated Evaluation of School Children Essays in Arabic
URI https://dx.doi.org/10.1016/j.procs.2017.10.089
https://doi.org/10.1016/j.procs.2017.10.089
UnpaywallVersion publishedVersion
Volume 117
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAFT
  databaseName: Open Access Digital Library
  customDbUrl:
  eissn: 1877-0509
  dateEnd: 99991231
  omitProxy: true
  ssIdentifier: ssj0000388917
  issn: 1877-0509
  databaseCode: KQ8
  dateStart: 20100501
  isFulltext: true
  titleUrlDefault: http://grweb.coalliance.org/oadl/oadl.html
  providerName: Colorado Alliance of Research Libraries
– providerCode: PRVESC
  databaseName: ScienceDirect Open Access Journals (Elsevier)
  customDbUrl:
  eissn: 1877-0509
  dateEnd: 99991231
  omitProxy: true
  ssIdentifier: ssj0000388917
  issn: 1877-0509
  databaseCode: IXB
  dateStart: 20100501
  isFulltext: true
  titleUrlDefault: https://www.sciencedirect.com
  providerName: Elsevier
– providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 1877-0509
  dateEnd: 99991231
  omitProxy: true
  ssIdentifier: ssj0000388917
  issn: 1877-0509
  databaseCode: M~E
  dateStart: 20100101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
– providerCode: PRVLSH
  databaseName: Elsevier Journals
  customDbUrl:
  mediaType: online
  eissn: 1877-0509
  dateEnd: 99991231
  omitProxy: true
  ssIdentifier: ssj0000388917
  issn: 1877-0509
  databaseCode: AKRWK
  dateStart: 20100501
  isFulltext: true
  providerName: Library Specific Holdings
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1bS8MwFA5jPuiLd3FeRh58tG7pNX2sY2M6HKIO51NJTlOojG64Dtm_N6c3FGSIjw05TfmSnpOEc76PkKvI94TDgBtccSzJYcKQMY8NwMoZW4Lnxljg_DB2hxP7fupMG6RX1cJgWmXp-wufnnvrsqVTotlZJEnnmXHPQ_YS5qHYjoucoJbNUb7hbnpb37Mg24mfC-9ifwMNKvKhPM0L4wTSdjPvBrO8UO799wC1vUoXYv0pZrNvAWiwT3bLnSMNio87IA2VHpK9SpWBlj_pEeHBKpvrfaiKaL-m8qbzmBaEm7RXlm_T_nIp1kuapPqdQiZwTCaD_ktvaJT6CAZYPssMy4p0cBbcFk4UR7Ib2cLsgj5AAQPgXWUqR0lQHmjIfQ0D4L2k9ESsZwN8blonpJnOU3VKqNKGrmQKYiu2XcV8fWpTrg-R6bsSTNUiZgVKCCV5OGpYzMIqS-w9zJEMEUls1Ei2yHVttCi4MzZ3dyu0wx9LINTefbOhUc_NXwY6--9A52QHn4obmAvSzD5W6lLvSTLZJlvB6Ol11M4Xn36ajB-Dty9c0uLQ
linkProvider Elsevier
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LTwIxEG4QD3jxbcRnDx6t0H12j0gwqMBFSLht2tlugiELkSWGf29nX9HEEOO1u7PdfG1nppOZbwi5iwJfuhwEE1pgSQ6XTMUiZoCVM44C34uxwHk48voT52XqTmukW9bCYFploftznZ5p62KkVaDZWs5mrTcufB_ZS7iPzXY8Z4fsOq7xTrCKb_pYBVqQ7iTIOu-iAEOJkn0oy_NCQ4G83dx_wDQv7Pf-u4VqrJOl3HzK-fybBXo6JPuF60g7-d8dkZpOjslB2ZaBFqf0hIjOOl0YR1RHtFdxedNFTHPGTdot6rdpb7WSmxWdJeabUs3glEyeeuNunxUNEhjYAU-ZbUfGOkvhSDeKI9WOHGm1wdyggAOItra0qxVoHwzmgYEBMDCpfBmb5YBAWPYZqSeLRJ8Tqo2gp7iG2I4dT_PAXNu0F0BkBZ4CSzeJVYISQsEejk0s5mGZJvYeZkiGiCQOGiSb5L4SWubkGdtf90q0wx97IDTqfbsgq9bmLxNd_HeiW9Loj4eDcPA8er0ke_gkD8dckXr6sdbXxkFJ1U22Ab8AoZriow
linkToUnpaywall http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA66HTw5f-JEJQePZqy_0uQ4xsYQHB4czFNJXhOYjm7YFpl_vcmaDhUd81ry2vJe2ve99n3fQ-g25bGIPGCEKWYpOZ4gUjNNwDJnQgkx1Zbg_DCmo0l4P42mTmfbcmG-_b9f92HZF7nV1fbijm3DYnwfNWlkgHcDNSfjx96zLalYHBOrZFLrCv1u-VfuOSizpVi9i_n8S24ZtirSdr6WJLQtJa-dspAd-Pgh2LjjbR-hQ4cxca_aFMdoT2UnqFXPb8DucT5FrFcWC4NYVYoHG9FvvNC4kubEfUf0xoM8F6sczzJzTiFncIYmw8FTf0TcJAUCAfcKEgSpSeOChSJKdSq7aSj8LphSCzwA1lW-ipQEFYMJDg8YA_sFU8ZCm7gBZ35wjhrZIlMXCCtjSKWnQAc6pMrjpr5TlEPqcyrBV23k1z5OwMmM22kX86TuJ3tJ1r5JrG_sQeObNrrbGC0rlY3ty2kdvMQBhQoAJCYO2w3JJtS7XOjyn-uvUKN4K9W1wSmFvHH78xNzQObM
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Automated+Evaluation+of+School+Children+Essays+in+Arabic&rft.jtitle=Procedia+computer+science&rft.au=Al-Jouie%2C+Maram+F.&rft.au=Azmi%2C+Aqil+M.&rft.date=2017&rft.issn=1877-0509&rft.eissn=1877-0509&rft.volume=117&rft.spage=19&rft.epage=22&rft_id=info:doi/10.1016%2Fj.procs.2017.10.089&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_procs_2017_10_089
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1877-0509&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1877-0509&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1877-0509&client=summon