COVID-19 Ad-hoc Remote Learning – Quality Assessment, “Seven Stars” Analysis and Lessons Learned

Document Type : Original Article

Author

School of Engineering and Technology Badr University at Cairo Cairo, Egypt

Abstract

This paper presents a novel metric to assess
remote learning. It demonstrates an analysis of 4000+ hours
of Nile University in addition to 15,000+ hours of YouTube
courses. The results validate the requirement of training of
remote learning delivery. It also evidences the lack of ILO
alignment between the Courses and the programs.

Highlights

A novel metrics is proposed to evaluate and rank remote
learning session. It consists of 7 criteria where tools are
used to automatically extract the rating of each session. A
protype has been developed and analyzed 3452 video
recordings that consists of more than 4000+ hours of
learning collected through Stream of Nile University.
Quality Assessment and lessons learned is summarized
as the Maximum, Minimum, Average, Median and
Standard Deviation of the 7 stars for 3542 video lectures.
Each star can have a value from 0 to 10. The distribution
and the percentage of the star values for 3542 video lectures
demonstrate the practicality of the approach.

Keywords


COVID-19 epidemic creates scenarios where students
and instructors, or resource, are not physically collocated in
a traditional classroom environment. Learning and
education activities are is relayed through technology,
remote access, discussion boards, video conferencing, and
may be online assessments.
The variability of engagement is a major concern as
remote learning may exacerbate social inequalities [1], [2]
Students from lower socioeconomic are less likely to have
online classes from their schools than their peers from
higher socioeconomic backgrounds (~32% compared to
~43% for primary, ~40% compared to ~58% for secondary)
[1]. Digital technology is not available to all students. In
addition, students may be deprived from access due to
financial or location limitation. Members with limited
access to digital technology or form lower-income families
are less likely to engage with remote learning [1][3]. More
generally, the scale and duration of school closures are
unprecedented, and the potential short- and longer-term
consequences for all students is a concern. The model of
remote education is predicted to stay beyond the virus era
[4].
School closures disadvantage children’s educational and
social development [5]. School narrows these gaps but the
gap widens over the long summer holiday [6]. There is a
danger that these school closures could reverse this
progress, with the median estimate indicating the gap could
widen by 36% [7]. The period between Key stage 2 (aged
7–8 years) and Key stage 4 (aged 15–16 years) is identified
as a crucial time to ensure that higher-attaining students
from lower socioeconomic backgrounds remain on the high
achievement trajectory [8].
The quality of the learning experience will have varied.
In a teacher survey on March 22, 2020, 6360 teachers
responded to indicate that for state primary schools; 51%
would be teaching using an online digital platform, with
49% using take-home physical worksheets or books. At
state secondary, this was 82% and 19% respectively [9].
Lack of access to computers, books, learning space or other
enriching opportunities create a home learning environment
that is less conducive to learning [10][11][12][13].
Governments have plans to provide laptop computers to
students from disadvantaged backgrounds
[14][15][16][17][18][19][20] A survey of 1784 children in
China after home confinement for a month found that
22·6% reported depressive symptoms [21], higher than
previously identified (17·2%) [22]. Up to 40% of children
receive special educational needs and disability support at
some point across the child life course [23]. Social
isolation, increased stress on parents and caregivers,
uncertain access to food and housing, loss of financial
income or support and worries about making ends meet
may carry higher risks for child abuse [24]. The lack of
outside contact with teachers, healthcare professionals and
social workers, raises the risk of missing opportunities to
identify early signs of the need for safeguarding
[25][26][27][28]
Frameworks for eLearning quality has been proposed
[29] including many assessment for online eLearning
[30][31][32][33][34][35][36]. However, none of them
produces a framework to analyze or assess the ad-hoc
learning. The adhoc remote learning produces am
environment with a unique opportunity to collect recording
of all courses, labs and learning pieces. These recordings
allow the implementation of a model and metrics to
measure the quality of the learning along with a set of tools
to automatically assess the Quality of the learning records.
Furthermore, to assess the quality of learning offered to
achieve the Intended Learning Outcomes (ILO).
This reset of this paper is structured as follows: Section
II demonstrate the proposed metrics to measure the quality
of o the online education. Section III shows the prototype
and the results. Section IV concludes this paper.
 
[1] Andrew, A., Cattan, S., Costa-Dias, M., Farquharson, C.,
Kraftman, L., Krutikova, S., Phimister, A. and Sevilla, A.
Learning during the Lockdown: Real-Time Data on
Children’s Experiences during Home Learning (IFS Briefing
Note BN288) [online]. Available:
https://www.ifs.org.uk/uploads/BN288-Learning-during-thelockdown-
1.pdf [10 July, 2020].
[2] Department for Education. ‘Billion pound Covid catch-up
plan to tackle impact of lost teaching time’, 19 June 2020
[online]. Available:
https://www.gov.uk/government/news/billion-pound-covidcatch-
up- plan-to-tackle-impact-of-lost-teaching-time [10
July, 2020.
[3] Lucas, M., Nelson, J. and Sims, D.. Schools’ Responses to
Covid-19: Pupil Engagement in Remote Learning [online].
Available:
https://www.nfer.ac.uk/media/4073/schools_responses_to_c
ovid_19_ pupil_engagement_in_remote_learning.pdf [10
July, 2020].
[4] Lee Gardner, “Covid-19 Has Forced Higher Ed to Pivot to
Online Learning. Here Are 7 Takeaways So Far,” The
Chronicle of Higher Education, MARCH 20, 2020
Available: https://www.chronicle.com/article/Covid-19-Has-
Forced-Higher-Ed/248297?cid=wcontentgrid_hp_1b
[5] K.L. Alexander, D.R. Entwisle, L.S. Olson, ”Lasting
consequences of the summer learning gap: Am. Socio. Rev.,
72 (2) (2007 Apr), pp. 167-180
[6] J. Shinwell, M.A. Defeyter, “Investigation of summer
learning loss in the UK-implications for holiday club
provision” Front Public Health, 5 (2017), p. 270
[7] Education Endowment Foundation, “Best evidence on
impact of school closures on the attainment gap” Available
from:https://educationendowmentfoundation.org.uk/covid-
19-resources/best-evidence-on-impact-of-school-closureson-
the-attainment-gap/ (2020)
[8] C. Crawford, L. Macmillan, A. Vignoles, Progress made by
high-attaining children from disadvantaged backgrounds
[internet]. London: social mobility and child poverty
commission Available
from:https://assets.publishing.service.gov.uk/government/up
loads/system/uploads/attachment_data/file/324501/High_att
ainers_progress_report_final.pdf (2014)
[9] Teacher Tapp, “What does distance learning look like in
England? (And where will teachers’ kids be today?)”
Available from: https://teachertapp.co.uk/what-doesdistance-
learning-look-like-in-england-and-where-willteachers-
kids-be-today/ (2020)
[10] H. Chowdry, C. Crawford, L. Dearden, R. Joyce, L. Sibieta,
K. Sylva, et al. Poorer children’s educational attainment:
how important are attitudes and behaviour? Joseph
Rowntree Foundation (2010) Available from:
https://www.jrf.org.uk/report/poorer-children’seducational-
attainment-how-important-are-attitudes-andbehaviour
[11] B. Shaw, S. Baars, L. Menzies, M. Parameshwaran, R. Allen
Low Income Pupils’ Progress at Secondary School
[Internet]Social Mobility Commission, London (2017)
Available
from:https://assets.publishing.service.gov.uk/government/up
loads/system/uploads/attachment_data/file/594363/Progress
_at_Secondary_School_report_final.pdf
[12] L. Dearden, L. Sibieta, K. Sylva The Socio-Economic
Gradient in Early Child Outcomes: Evidence from the
Millennium Cohort Study. LLCS [Internet]. 2011 Jan 31
[cited 2020 May 22];2(1). Available
from:http://www.llcsjournal.org/index.php/llcs/article/view/
140
[13] C. Cullinane, R. Montacute COVID-19 and social mobility
impact brief #1: school shutdown The Sutton Trust;
Available from:https://www.suttontrust.com/wpcontent/
uploads/2020/04/COVID-19-Impact-Brief-School-
Shutdown.pdf (04/2020)
[14] Department for Education Get technology support for
children and schools during coronavirus (COVID-
19)Available from:https://www.gov.uk/guidance/get-helpwith-
technology-for-remote-education-during-coronaviruscovid-
19 (2020 Apr)
[15] F. Whittaker, Coronavirus: majority of free laptops won’t be
delivered until June. Schools Week [Internet] Available
from:https://schoolsweek.co.uk/coronavirus-majority-offree-
laptops-wont-be-delivered-until-june/ (2020 Apr 29)
[16] Department for Education, Devices and 4G wireless routers
data Ad-hoc Notice: laptops, tablets and 4G wireless routers
for disadvantaged and vulnerable children: progress data
[Internet] Available from:
https://assets.publishing.service.gov.uk/government/uploads/
system/uploads/attachment_data/file/892510/Devices_and_4
G_wireless_routers_data_ad_hoc_stats.pdf (2020 Jun)
[17] Joseph Rowntree Foundation UK poverty 2019/20 (2020)
[18] Department for Education, Providing free school meals
during the coronavirus outbreak Available from:
https://www.gov.uk/government/publications/covid-19-freeschool-
meals-guidance/covid-19-free-school-mealsguidance-
for-schools (2020 May)
[19] The Food Foundation, Three million people are going
hungry just three weeks into lockdown new survey results
released, Available from: https://foodfoundation.org.uk/wpcontent/
uploads/2020/04/Hunger-release-FINAL.pdf (2020)
[20] T. Ford, R. Goodman, H. Meltzer, The British child and
adolescent mental health survey 1999: the prevalence of
DSM-IV disorders, J. Am. Acad. Child Adolesc. Psychiatr.,
42 (10) (2003 Oct), pp. 1203-1211
[21] X. Xie, Q. Xue, Y. Zhou, K. Zhu, Q. Liu, J. Zhang, et al.
Mental health status among children in home confinement
during the coronavirus disease 2019 outbreak in hubei
province, China JAMA Pediatr. (2020 Apr 24) Available
from
https://jamanetwork.com/journals/jamapediatrics/fullarticle/
2765196
[22] D.-D. Xu, W.-W. Rao, X.-L. Cao, S.-Y. Wen, F.-R. An, W.-
I. Che, et al. Prevalence of depressive symptoms in primary
school students in China: a systematic review and metaanalysis
J. Affect. Disord., 268 (2020 May), pp. 20-27
[23] M.A. Jay, R. Gilbert Special educational needs, social care
and health Arch. Dis. Child. (2020 Jan 22) archdischild-
2019-317985
[24] C.M. Rosenthal, L.A. Thompson Child abuse awareness
month during the coronavirus disease 2019 pandemic JAMA
Pediatr [Internet] (2020 Apr 24); Available from:
http://jamanetwork.com/journals/jamapediatrics/fullarticle/2
765195
[25] P. Green Risks to children and young people during covid-
19 pandemic, BMJ (2020 Apr 28), p. m1669
[26] Children’s Commissioner Childhood vulnerability in
England 2019 Available from:
https://www.childrenscommissioner.gov.uk
/publication/childhood-vulnerability-in-england-2019/
(2019)
[27] Assche, F.v. & R. Vuorikari 2006. A framework for quality
of learning resources. In: Ehlers, U.-D. & J.M. Pawlowski
(Eds.) Handbook on Quality and Standardisation in Elearning.
Springer, Berlin/ Heidelberg New York.
[28] Berger, T. & U. Rockmann, 2006. Quality of e-learning
products. In: Ehlers, U.-D. & J.M. Pawlowski (Eds.)
Handbook on Quality and Standardisation in E-learning.
Springer, Berlin/ Heidelberg New York.
[29] CEN/ ISS. WS-LT. CWA 14926. 2004. Guidelines for the
production of learner information standards and
specifications. Brussels.
[30] Dillon, P.; Posser D. & T. Howe, 2004. Design Transactions
in educational multimedia.The Design Journal, 7(2).
[31] Donabedian, A., 1980. Explorations in Quality Assessment
and Monitoring. Ann Arbor, Health Administration Press.
[32] Dondi, C., Moretti, M. & F. Nascimbeni, 2006. Quality of elearning:
Negotiating a strategy, implementing a policy. In:
Ehlers, U.-D. & J.M. Pawlowski (Eds.) Handbook on
Quality and Standardisation in E-learning. Springer, Berlin/
Heidelberg NY.
[33] Duval, E., 2006. LearRank: Toward a real quality measure
for learning.. In: Ehlers, U.-D. & J.M. Pawlowski (Eds.)
Handbook on Quality and Standardisation in E-learning.
Springer, Berlin/ Heidelberg New York.
[34] Ehlers, U.-D. 2006. Myths and realitites in learner oriented
e- learning quality. In: Ehlers, U.-D. & J.M. Pawlowski
(Eds.) Handbook on Quality and Standardisation in Elearning.
Springer, Berlin/ Heidelberg New York.
[35] Ehlers, U.-D. & L. Goertz, 2006. Quality evaluation for elearning
in Europe. In: Ehlers, U.-D. & J.M. Pawlowski
(Eds.) Handbook on Quality and Standardisation in Elearning.
Springer, Berlin/ Heidelberg New York.
[36] Ehlers, U.-D. & J.M. Pawlowski, 2006. Quality in European
e- learning: An introduction. In: Ehlers, U.-D. & J.M.
Pawlowski (Eds.) Handbook on Quality and Standardisation
in E-learning. Springer, Berlin/ Heidelberg New York.