際際滷

際際滷Share a Scribd company logo
Evaluation of
Learners Experiences of
E-learning
SIG
#ELESIG
is a community of researchers and
practitioners from HE and FE.
involved in investigations of learners'
experiences and uses of technology in
learning.
work together to share knowledge and
practice and develop a shared repertoire
of resources
ELESIG North West
Tunde
Varga-Atkins
Jim
Turner
Claire
Moscrop
Roger
Harrison
Natasa
Lackovic
University of
Manchester
Lancaster
University
Edge Hill
University
LJMU University of
Liverpool
Researching learners' experiences and uses of
technology using action research #LERMOOC
Cityscape as metaphor of assessment practice: report from
an evaluation of an e-submission & e-marking project
T端nde Varga-Atkins, University of Liverpool
Policy status: University of Liverpool 2016/17
e-submission: requirement*
e-marking: encouraged
phased, supported approach
*with exceptions where not possible
Data collection
1. Staff surveys (n=345) and interviews
2. Student enhancement projects & JISC
Digital tracker student survey (n=670)
3. Benchmarking study
cc: jeffmason - https://www.flickr.com/photos/23412868@N03
Places: making meaning of e-submission and e-marking
Cityscape:
assessment practice
Greenhouse:
transparency of
team-marking
Sliding doors:
feedback dialogue
Cityscape
cc: identity chris is - https://www.flickr.com/photos/95579828@N00
cc: Fluffymuppet - https://www.flickr.com/photos/75774141@N00
greenhouse
cc: AntoineMeu - https://www.flickr.com/photos/56321333@N05
sliding doors
Feedback space
Student more divided on eMarking (n=670)
cc: AntoineMeu - https://www.flickr.com/photos/56321333@N05
sliding doors
cc: Magdalena Roeseler - https://www.flickr.com/photos/84894254@N03
digital placemaking
What you can do
 Sign up to elesig.net and join the
ELESIG NW group on the ELESIG
site elesig.net
 Advertise ELESIG NW in your
institution.
 Use #elesig
 Look out for our #LERMOOC
 Join us as a committee member.
Thank you ------ with acknowledgements
 Daniel Roberts, Alex Spiers, Ben McGrae, Nick Bunyan,
Andy Dolben, Lynn Rendon for contribution to the
Evaluation Report
 JISC EMA readiness survey (2 questions used in our
survey)
 JISC digital tracker survey (2 questions related to e-
submission and e-feedback)
tva@liverpool.ac.uk
@tundeva

More Related Content

Cityscape as metaphor of assessment practice: report from an evaluation of an e-submission & e-marking project

  • 1. Evaluation of Learners Experiences of E-learning SIG #ELESIG is a community of researchers and practitioners from HE and FE. involved in investigations of learners' experiences and uses of technology in learning. work together to share knowledge and practice and develop a shared repertoire of resources
  • 2. ELESIG North West Tunde Varga-Atkins Jim Turner Claire Moscrop Roger Harrison Natasa Lackovic University of Manchester Lancaster University Edge Hill University LJMU University of Liverpool
  • 3. Researching learners' experiences and uses of technology using action research #LERMOOC
  • 4. Cityscape as metaphor of assessment practice: report from an evaluation of an e-submission & e-marking project T端nde Varga-Atkins, University of Liverpool
  • 5. Policy status: University of Liverpool 2016/17 e-submission: requirement* e-marking: encouraged phased, supported approach *with exceptions where not possible
  • 6. Data collection 1. Staff surveys (n=345) and interviews 2. Student enhancement projects & JISC Digital tracker student survey (n=670) 3. Benchmarking study cc: jeffmason - https://www.flickr.com/photos/23412868@N03
  • 7. Places: making meaning of e-submission and e-marking Cityscape: assessment practice Greenhouse: transparency of team-marking Sliding doors: feedback dialogue
  • 9. cc: identity chris is - https://www.flickr.com/photos/95579828@N00
  • 10. cc: Fluffymuppet - https://www.flickr.com/photos/75774141@N00 greenhouse
  • 11. cc: AntoineMeu - https://www.flickr.com/photos/56321333@N05 sliding doors
  • 12. Feedback space Student more divided on eMarking (n=670)
  • 13. cc: AntoineMeu - https://www.flickr.com/photos/56321333@N05 sliding doors
  • 14. cc: Magdalena Roeseler - https://www.flickr.com/photos/84894254@N03 digital placemaking
  • 15. What you can do Sign up to elesig.net and join the ELESIG NW group on the ELESIG site elesig.net Advertise ELESIG NW in your institution. Use #elesig Look out for our #LERMOOC Join us as a committee member.
  • 16. Thank you ------ with acknowledgements Daniel Roberts, Alex Spiers, Ben McGrae, Nick Bunyan, Andy Dolben, Lynn Rendon for contribution to the Evaluation Report JISC EMA readiness survey (2 questions used in our survey) JISC digital tracker survey (2 questions related to e- submission and e-feedback) tva@liverpool.ac.uk @tundeva

Editor's Notes

  • #3: If you are interested in getting involved in ELESIG NW either as a member or a committee member. Chat to one of us, or sign up to elesig.net or find us on social media
  • #4: We also ran a MOOC (mini-MOOC) this year on learner exp research (with technology)
  • #5: Learnt this from Alex: choosing an image to depict or represent a project or findings for easy institutional communication. We choose this image before we knew about this event on digital placemaking! So very apt. Todays presentation Example of an ELESIG-type research (but also staff perspective) Highlighting spatial aspects of e-marking and e-submission study In this short time, I decided to focus on 3 aspects that relates to place (40-page report!!) In a way I inverted digital place: from this studys findings about digital practices = I have found metaphors of place to give insight about them Meaning-making: from space to place
  • #6: e-marking Schools and departments at various stages: some all e-marking, others piloted it this year, others not yet.
  • #8: More detail consult full report. Cityscape metaphor for assessment practice Greenhouse transparency of team-marking Sliding doors places for feedback dialogue For each: I will give the metaphor = the place Explain how/why it relates to in terms of e-submission and e-marking) Evidence or example from our findings Implication for the university
  • #9: Metaphor = On choosing this city-scape image : no building is the same as any other in the city. Explanation with re to e-marking= The mix of traditional, established buildings existing alongside newer, modern buildings. This seems an apt metaphor for representing how different districts, or Schools (some with adopting institutional, others their own policies for eMarking), all come together in a multifaceted cityscape. Staff are influenced by a number of contextual factors in deciding whether to upgrade or continue with established ways of marking. Also captures that whilst with the evaluation we were looking for trends and generalisations, summaries the finding is really that e-marking practices are highly individual with some elements of uniformity (as by district or building clusters). Appendix: Place: cityscape , Wikipedia = Inurban designthe terms refer to the configuration of built forms and interstitial space. (built forms and spaces between structures/buildings), Another way to use an interstitial space is to incorporate a design that divides the functions of the building into groups and localizes them.
  • #10: Lets zoom in to the cityscape a bit more to give you an example. Example evidence = lecturers may be from same School, having access to the same tools (tablets, Ipad Pro) but prefer to use different devices for marking, or prefer to mark at different places or have different assessments, module sizes and teams, use different tools (Bb, Tii, PebblePad, online tests etc) Devices used for e-marking: desktop and dual screens still dominate (despite some tablet pilots) = more functionality Terraced houses here: may look the same but not the same inside! Nuanced differences and nuances in practice, needs etc. Implication: challenge to uniform(ise) (hence the supported, local approach rather than central requirement without negotiation) BUT good practices and joint training, involvement of assessment officer and local support helps!
  • #11: Not using this slide but links to cityscape
  • #12: This time I start with relating the finding on e-marking to the metaphor. The survey elicited views on benefits of e-marking Convenience for students and staff cited as top two, then legibility Where there were departmental or School-wide e-marking requirements there seemed to be benefits in terms of the quality of feedback = resulted in better transparency, consistency of feedback This was because staff were able to log on to the VLE and see how others have marked and what feedback they gave to students. Chosen metaphor: greenhouse: you can see what the other gardeners are doing, how they are nurturing the plants, how much water they give and so on. As a staff member Implication = one aspect of e-marking workflow that enables the production of better feedback and quality control by module leader (in case of multiple markers).
  • #13: The third key finding was about student preferences with respect to digital feedback. Sliding doors (film reference although havent actually seen the film) two different stories depending on whether someone can get on the subway or not, whether the doors are open or not. Also about the expectation of whether and when the doors will open and close. What it refers to in e-marking: is there an opportunity for students to engage with feedback (open door) or not (closed door) Students were asked if they engage with feedback more if it is delivered digitally. 50% of students said yes (potentially bec they can access it conveniently, immediately.) The rest were more divided. This has also come up from staff. In essence, students do not mind form of feedback as long as it is good quality. However, there are some concerns about loss of interaction, students feeling they cannot discuss feedback with tutor; tool doesnt offer dialogue option. Staff need to be mindful for keeping these spaces open where students-staff can talk over the assignment and discuss the feedback. Sliding doors when is it shut and when is it open? Would students miss this opportunity? Do they know if the feedback is digital whether the door is still open to them or not to discuss in person? Is a feedback place still created? Implication: staff needs to communicate this to students to let students know that if even if feedback is digitally accessible, do they have opportunities to discuss together (some do, some organise such sessions and have printed Tii forms with them etc.)
  • #14: Student views from the JISC Digital Tracker survey (n=670 Liverpool students) were more divided with regards to eMarking than with eSubmission. The quality of feedback seems more important to students than the mode through which it is delivered. About one in five disagree with the statement that they make more use of feedback if delivered digitally. There could be a number of reasons for this. For instance, it could be about Turnitin/tools not enabling a feedback dialogue between staff and students, or that students prefer collecting feedback from staff in person. The Student Report will illuminate this survey finding on eFeedback in more detail. Still, nearly 50% of students say that they engage more with feedback when delivered digitally. It certainly points to the importance of the quality of feedback over the mode through which feedback is delivered. This confirms the staff survey: many [students] prefer to receive hard copy feedback. Of course, this will take a while to bed in amongst our students but they certainly do not see electronic feedback as the 'best' way to receive feedback at present. For almost half of the students, eFeedback does however offer more opportunities for engagement.
  • #15: The third key finding was about student preferences with respect to digital feedback. Sliding doors (film reference although havent actually seen the film) two different stories depending on whether someone can get on the subway or not, whether the doors are open or not. Also about the expectation of whether and when the doors will open and close. What it refers to in e-marking: is there an opportunity for students to engage with feedback (open door) or not (closed door) Students were asked if they engage with feedback more if it is delivered digitally. 50% of students said yes (potentially bec they can access it conveniently, immediately.) The rest were more divided. This has also come up from staff. In essence, students do not mind form of feedback as long as it is good quality. However, there are some concerns about loss of interaction, students feeling they cannot discuss feedback with tutor; tool doesnt offer dialogue option. Staff need to be mindful for keeping these spaces open where students-staff can talk over the assignment and discuss the feedback. Sliding doors when is it shut and when is it open? Would students miss this opportunity? Do they know if the feedback is digital whether the door is still open to them or not to discuss in person? Is a feedback place still created? Implication: staff needs to communicate this to students to let students know that if even if feedback is digitally accessible, do they have opportunities to discuss together (some do, some organise such sessions and have printed Tii forms with them etc.)
  • #16: What digital placemaking means to me Each presenter during the day will offer a response to this question and colleagues connecting into the conference will also give us their thoughts. For me today: Place is space with given meaning In a way I inverted digital place: I have found metaphors of spaces/places to give insight to the studys key findings about digital practices A related observation is that although Digital can expand time & space access to assignments & feedback anytime anywhere (staff and students) BUT we also need to be mindful of sliding doors = that we are not shutting off dialogue off managing our sliding doors There are digital (or real) places still available for feedback dialogue (Minkovski spacetime is created!!)