From 89d536d3d071e33cd8e6d4a1465a36f265b46586 Mon Sep 17 00:00:00 2001 From: pattydl Date: Fri, 2 May 2025 17:40:32 -0600 Subject: [PATCH] Adding comments to plays in EGP playgook --- .../100-grading-assignments/010-continuous-rubric.md | 2 ++ .../100-grading-assignments/020-adaptive-peer-grading.md | 1 + .../100-grading-assignments/030-credit-for-completion.md | 2 ++ .../100-grading-assignments/040-grading-group-work.md | 2 ++ .../100-grading-assignments/050-molecular-biology-TBL.md | 8 ++++++++ _chapters/100-grading-assignments/090-slide-rubrics.md | 2 ++ .../010-differentiated_instruction_pre_and_post_tests.md | 2 ++ _chapters/200-grading-courses/020-techniques-grading.md | 2 ++ _chapters/200-grading-courses/030-specs-plus-equity.md | 2 ++ .../040-ungrading-no-instructor-grades.md | 2 ++ ...lop-soft-skills-without-including-them-in-the-grade.md | 2 ++ .../010-class-reflections-for-getting-buy-in.md | 2 ++ .../020-emphasize-feedback-and-reflection.md | 2 ++ .../030-set-reform-conditions.md | 2 ++ .../500-adapting-elearning-tools/010-autograding-CS.md | 2 ++ _chapters/800-Incubator/280-multiple-assessments.md | 2 ++ _chapters/800-Incubator/300-pre-course-interviews.md | 2 ++ 17 files changed, 39 insertions(+) diff --git a/_chapters/100-grading-assignments/010-continuous-rubric.md b/_chapters/100-grading-assignments/010-continuous-rubric.md index 440b08c..c837eab 100644 --- a/_chapters/100-grading-assignments/010-continuous-rubric.md +++ b/_chapters/100-grading-assignments/010-continuous-rubric.md @@ -61,3 +61,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +I had some difficulty finding the source article that included Figure 2 (it was not available through my institution). I found it through EBSCO using my institution - https://openurl.ebsco.com/EPDB%3Agcd%3A12%3A9432796/detailv2?sid=ebsco%3Aplink%3Ascholar&id=ebsco%3Agcd%3A112358532&crl. diff --git a/_chapters/100-grading-assignments/020-adaptive-peer-grading.md b/_chapters/100-grading-assignments/020-adaptive-peer-grading.md index a1c10f1..87fbdae 100644 --- a/_chapters/100-grading-assignments/020-adaptive-peer-grading.md +++ b/_chapters/100-grading-assignments/020-adaptive-peer-grading.md @@ -67,3 +67,4 @@ See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. +Suggestion: Another approach to peer grading might be anonymous posting of all assignments, allowing students to see a variety of work and picking 3 assignments to give feedback on, perhaps one in each area (High/Distinction, Credit/Pass, and Fail). This gives a student a broader scope of work. diff --git a/_chapters/100-grading-assignments/030-credit-for-completion.md b/_chapters/100-grading-assignments/030-credit-for-completion.md index e3ad874..127be01 100644 --- a/_chapters/100-grading-assignments/030-credit-for-completion.md +++ b/_chapters/100-grading-assignments/030-credit-for-completion.md @@ -38,3 +38,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +This play may be more suitable for formative assessments like chapter quizzes, in-class activities and discussions, and other low-stakes grading assignments. \ No newline at end of file diff --git a/_chapters/100-grading-assignments/040-grading-group-work.md b/_chapters/100-grading-assignments/040-grading-group-work.md index 1945e73..9e7d10b 100644 --- a/_chapters/100-grading-assignments/040-grading-group-work.md +++ b/_chapters/100-grading-assignments/040-grading-group-work.md @@ -81,3 +81,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +The challenge with this play is that with a traditional grading scheme, students may not be as likely to collaborate and share information if there is a limit on the number of students that get the top grades. In an A’s for All environment proposed by Dan Garcia and Armando Fox, this challenge is unlikely to apply. See Hayek et al, (2015). Grading Hampers Cooperative Information Sharing in Group Problem Solving“; in Social Psychology:Vol. 46(3):121–131, DOI: 10.1027/1864-9335/a000232. \ No newline at end of file diff --git a/_chapters/100-grading-assignments/050-molecular-biology-TBL.md b/_chapters/100-grading-assignments/050-molecular-biology-TBL.md index 571f4a6..8860896 100644 --- a/_chapters/100-grading-assignments/050-molecular-biology-TBL.md +++ b/_chapters/100-grading-assignments/050-molecular-biology-TBL.md @@ -59,3 +59,11 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +The Affinity Research Group Model developed by the Computing Alliance of Hispanic Serving Institutions (CAHSI) cites five basic elements critical to team-based learning: +1. Positive interdependence +2. Face-to-face promotive interaction +3. Individual accountability +4. Group and professional skills +5. Group processing +An example three-person team activity is to perform a task (counting triangles in a drawings) individually and then as a team (two different drawings of course), and compare results. In the team activity, each team member has a role - Recorder: Documents process, Checker of Understanding: Ensures all members understand, Participation Checker: Ensures that all members are actively participating. Roles get rotated and team members also rotate on different assignments so as to mix up the teams as appropriate. See Kephart, Villa, Gates, and Roach (2008) – “The Affinity Research Group Model: Creating and Maintaining Dynamic, Productive, and Inclusive Research Groups”. diff --git a/_chapters/100-grading-assignments/090-slide-rubrics.md b/_chapters/100-grading-assignments/090-slide-rubrics.md index 68352e9..9d5aba0 100644 --- a/_chapters/100-grading-assignments/090-slide-rubrics.md +++ b/_chapters/100-grading-assignments/090-slide-rubrics.md @@ -60,3 +60,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +Most struggling students will only be motivated by moving up on a rubric – they will be demotivated by moving down on a rubric, especially if the assignment rubric is not well defined. Limiting how many bands a student could move seems biased if they truly make big improvments. It could be used instead as extra points used if a student is borderline for the final grade, and thus you might consider moving this play to the Grading Courses section. What is of real value in the slide rubrics play is feedback that is SMART and encourages them to try for the next band or two up (specific, measurable, actionable, relevant, and timely). \ No newline at end of file diff --git a/_chapters/200-grading-courses/010-differentiated_instruction_pre_and_post_tests.md b/_chapters/200-grading-courses/010-differentiated_instruction_pre_and_post_tests.md index 543f98e..b0decf9 100644 --- a/_chapters/200-grading-courses/010-differentiated_instruction_pre_and_post_tests.md +++ b/_chapters/200-grading-courses/010-differentiated_instruction_pre_and_post_tests.md @@ -61,3 +61,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +There are a few challenges with what has been presented. Pre-tests are reminiscent of placement exams, and depending on the instructor, the test, and the culture within the department, they might be used to weed students out or students may perceive them as such (my former department chair, now dean, has argued against them). Providing individualized instruction is not scalable for large classes like intro CS courses that are prerequisites for advancing in the degree. An alternative might be refresher course modules that pre-test as described above for knowledge/learning gaps that can bring students up to speed, then identifying via post-testing any concepts that are still not mastered. With a good test implementation, a student would only need to retake sections in which they did poorly. Ideally, students signed up for a course could take these refresher course modules before the start of the course. It might also be used as a positive indicator if a final grade is borderline. diff --git a/_chapters/200-grading-courses/020-techniques-grading.md b/_chapters/200-grading-courses/020-techniques-grading.md index cb973b6..09b8e8e 100644 --- a/_chapters/200-grading-courses/020-techniques-grading.md +++ b/_chapters/200-grading-courses/020-techniques-grading.md @@ -76,3 +76,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here + +As a mastery-based approach for proof-intensive courses, this play would be optimally supported by lots and lots of examples and practice exercises that do NOT impact a student’s grade, while allowing them to get comfortable with the notation. For even stronger mastery, providing tools that support the ability to visualize the problem (mapping an unknown complexity problem to an NP-complete known problem, visualizing combinatorics problems, or analyzing program complexity). \ No newline at end of file diff --git a/_chapters/200-grading-courses/030-specs-plus-equity.md b/_chapters/200-grading-courses/030-specs-plus-equity.md index b100d35..8f6df89 100644 --- a/_chapters/200-grading-courses/030-specs-plus-equity.md +++ b/_chapters/200-grading-courses/030-specs-plus-equity.md @@ -88,3 +88,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +Are there are folks who support and use specs grading but are not bought into grading for equity? It seems like grading for equity incorporates many aspects of specs grading (using Talbert’s EMRN rubric or the GTMJ rubric introduced earlier). When combined with low-stakes assignments and regrading with SMART feedback whenever possible, I posit that grading for equity would generally result in stronger mastery than specs grading alone. It may be more challenging to implement and teach specs grading in a course that has been traditionally graded, especially if that course is a pre-requisite for continuing in the degree program or is an upper-level course required for graduation. If individual assignments are a binary pass/fail until mastery, student perceptions of success in the course may be impacted, so an instructor will have to continually socialize the message about mastery vs. grades. It also carries a risk for junior faculty whose tenure decision may depend on student evaluations. diff --git a/_chapters/200-grading-courses/040-ungrading-no-instructor-grades.md b/_chapters/200-grading-courses/040-ungrading-no-instructor-grades.md index 8e520c1..2f6cce7 100644 --- a/_chapters/200-grading-courses/040-ungrading-no-instructor-grades.md +++ b/_chapters/200-grading-courses/040-ungrading-no-instructor-grades.md @@ -53,3 +53,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +If one works in a department and culture that is holding onto traditional grading with a death grip, it would be difficult to contemplate piloting an ungraded course. It would be wiser to move along the spectrum from traditional grading to specs grading to grading for equity and then possibly ungrading for specific courses or assignments. This play might work best in a Senior Project/Capstone course with the socialized message that students need to learn how to assess their own work, since the technical workplace doesn’t give grades. Oftentimes new hires will be left completely on their own to navigate the hidden organizational culture for job opportunities, promotions, and alternate career paths that have been historically highly competitive and often lacking in mentorship, especially for marginalized employees. diff --git a/_chapters/200-grading-courses/050-develop-soft-skills-without-including-them-in-the-grade.md b/_chapters/200-grading-courses/050-develop-soft-skills-without-including-them-in-the-grade.md index ca99f8b..1b19258 100644 --- a/_chapters/200-grading-courses/050-develop-soft-skills-without-including-them-in-the-grade.md +++ b/_chapters/200-grading-courses/050-develop-soft-skills-without-including-them-in-the-grade.md @@ -74,3 +74,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +This is the play that I struggle the most with. Having spent 32 years working in tech, I would argue that “soft skills” are actually essential skills that are as important as technical skills, and are absolutely necessary to thrive in a professional work environment. While most tech companies currently interview for codin/whiteboarding skills, they often miss red flag behavior that can be discovered through behavioral interviews (I was a volunteer campus manager for my company for a decade). Companies are now less forgiving of essential skill gaps, and it has become more challenging for students to find and keep an entry level position. Essential skills must be developed alongside academic skills, whether they are graded or not. This is best practiced through group/team projects and senior capstone courses, which most students hate because of bad experiences with other students who take over, don't listen to them, exclude them, or contribute nothing. Many students who work, have transportation issues, familial or other non-school commitments, or scheduling conflicts may not be able to contribute outside of the classroom, so adequate time must be allowed for in-class group work. When students transition to the workplace, they are evaluated primarily on their soft skills as their technical skills develop. Perhaps this is an area where ungrading and peer evaluation plays can be combined to strengthen essential skills. I highly recommend developing an equitable team collaboration play that builds essential skills through group project work by structuring the group projects with a collaboration rubric, peer evaluations, ungrading, and criteria for team interactions that avoid many of the conflicts and issues that students report. \ No newline at end of file diff --git a/_chapters/400-communication-and-culture-change/010-class-reflections-for-getting-buy-in.md b/_chapters/400-communication-and-culture-change/010-class-reflections-for-getting-buy-in.md index 77c0eb0..036b16b 100644 --- a/_chapters/400-communication-and-culture-change/010-class-reflections-for-getting-buy-in.md +++ b/_chapters/400-communication-and-culture-change/010-class-reflections-for-getting-buy-in.md @@ -46,3 +46,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +I chose not to ask for permission to implement an equitable grading scheme as a new junior faculty as it was my research area, and my colleagues may disagree with me on whether I gave too many A’s in the two required courses I have taught thus far, but I was the first faculty in the deparment to implement several of Feldman's "Grading for Equity" interventions. I did constantly socialize my message that as a student, I had experienced inequitable grading, and as a CS educator, I chose to use an equitable grading scheme so that they could have a more positive educational experience. I repeatedly asked them to focus on mastery and learning, and that with some effort, their grades would reflect this. I socialized my messages in the course syllabus, including the statement “every student can earn an A” and that the bar for an A was not unreasonable. I built trust and minimized regrading by designing large automated question banks to allow for multiple resubmissions with flexible deadlines on quizzes and some assignments, even allowed a retake for a mid-term exam where students were very anxious about failing the course. Student reflections in group discussions are important for many reasons, not only to assess their own mastery and see their growth, but to build their competence and confidence in the subject matter, to build a sense of belonging by learning to listen and advocate their own views, and to know that not only did they do correct work, but they also did great work. I shared the learning outcomes we were covering at the start of each lecture slide deck, and ensured my decks were designed with accessibility in mind, including lots of examples; video recordings and transcripts were available online for students who might have missed a lecture. I also implemented an end-of-course student survey as a MS Forms mid-term survey given before the drop date, so that I could get feedback on my teaching and course-correct before the end of the semester. This enabled students to vent early and enabled me to get more positive student evaluations at a higher rate of participation at the end of the course. My total DFW rate for both courses was 2/81 students, and true to my word, most of the students earned A's and B's in both courses. \ No newline at end of file diff --git a/_chapters/400-communication-and-culture-change/020-emphasize-feedback-and-reflection.md b/_chapters/400-communication-and-culture-change/020-emphasize-feedback-and-reflection.md index 5162627..62a8a97 100644 --- a/_chapters/400-communication-and-culture-change/020-emphasize-feedback-and-reflection.md +++ b/_chapters/400-communication-and-culture-change/020-emphasize-feedback-and-reflection.md @@ -66,3 +66,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +I’ve stated this consistently throughout this playbook that SMART feedback, provided in an encouraging way, is key to student motivation, belonging, confidence, and competence. I also believe it is critical to share multiple career pathways so that students understand that they can have a successful career in computing besides the obvious coding ones. There are many needed skills besides coding proficiency, and I do my best to educate them on those that are a better fit for their skills, ethics, and interests. I also discuss what kind of company culture they want to be a part of so they choose employers and roles wisely. diff --git a/_chapters/400-communication-and-culture-change/030-set-reform-conditions.md b/_chapters/400-communication-and-culture-change/030-set-reform-conditions.md index 74ed642..fcc00c7 100644 --- a/_chapters/400-communication-and-culture-change/030-set-reform-conditions.md +++ b/_chapters/400-communication-and-culture-change/030-set-reform-conditions.md @@ -63,3 +63,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +My recommended approach to large-scale reform within a department or institution is through the use of EGP pilots for courses that have historically seen high DFW rates, especially in first and second year students. Within my own department, I have recommended that the department invest in curriculum and instruction redesign of courses that are typically assigned to new and/or junior faculty by incorporating EGP practices. New/junior faculty are typically given an existing course in Canvas that they may minimally modify their first semester teaching. Instead, by setting them up with the same course that has already been redesigned for EGP, not only do they have a better chance of retaining and progressing students, they also have been given EGP best practices that they can incorporate into any courses they design on their own. Ideally a department would staff a curriculum designer to create these Canvas development courses in collaboration with faculty whose research areas best match the course curriculum. diff --git a/_chapters/500-adapting-elearning-tools/010-autograding-CS.md b/_chapters/500-adapting-elearning-tools/010-autograding-CS.md index a6ccf79..1e0db53 100644 --- a/_chapters/500-adapting-elearning-tools/010-autograding-CS.md +++ b/_chapters/500-adapting-elearning-tools/010-autograding-CS.md @@ -48,3 +48,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +Automated grading is certainly the way to go, but it needs to be bulletproof, especially in introductory courses, and especially for coding assignments that are built-upon over time. I was advising a student who had fallen behind in a self-paced course that had automated the grading system – the course was a prerequisite to progress in the degree. I had the student walk me through how she was working through the coding assignment, which had a series of automated test data that were run to test her code. Each and every version of code had to be checked into GitHub before testing would proceed. What she did not understand was that she was breaking the code that had worked on earlier tests to pass later ones, and then the earlier tests were failing without telling her why. While she eventually understood this, she was already past the midterm stage and eventually failed the course and had to repeat it. Not only does automated grading need to be bulletproof, but students also need to be comfortable asking for help and receiving it in a timely manner. An AI assistant could be useful here, assuming it had specific knowledge of how the automated grading system worked and could coach a student through some of the rough spots or provide hints to lines of specific code that are problematic. In lieu of this, students should have ready resources to visit their instructor or TAs, and receive SMART feedback. In large introductory classes, this may be difficult to achieve. A fee-based automated grading tool like PrairieLearn would be great to adopt, but it may be cost-prohibitive for some public institutions. \ No newline at end of file diff --git a/_chapters/800-Incubator/280-multiple-assessments.md b/_chapters/800-Incubator/280-multiple-assessments.md index eb812f8..1b9b446 100644 --- a/_chapters/800-Incubator/280-multiple-assessments.md +++ b/_chapters/800-Incubator/280-multiple-assessments.md @@ -54,3 +54,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +I agree with solutions provided. I would add the use of academic controversy for current events and real-world problems (use of surveillance/face recognition, generation of deep-fake videos, social media harm, etc.) to build ethical leadership and advocacy skills. See Alvidrez et al (2024) “Using Academic Controversy in a Computer Science Undergraduate Leadership Course: An Effective Approach to Examine Ethical Issues in Computer Science”, 2023 ASEE Annual Conference & Exposition. \ No newline at end of file diff --git a/_chapters/800-Incubator/300-pre-course-interviews.md b/_chapters/800-Incubator/300-pre-course-interviews.md index 983e5e4..958a667 100644 --- a/_chapters/800-Incubator/300-pre-course-interviews.md +++ b/_chapters/800-Incubator/300-pre-course-interviews.md @@ -50,3 +50,5 @@ posting comments below. See {% include chapter-link.html slug="join-discussions" %} for details. * Insert a comment here. + +I’m not a fan of pre-course interviews, not only because of the time factor required to interview all students, but primarily because there is tremendous opportunity to introduce bias against a student’s perceived abilities that may persist throughout the course. Whuile we might know how we would conduct the interviews, depending on the department culture and implementation, they could be harmful. Interviews, like whiteboarding exercises (which are performance exercises and also very ableist!), can be highly subjective, and under stereotype threat, marginalized students may underperform in these interviews. Pre-course refresher assignments and activities to gain or regain learning loss are preferred. \ No newline at end of file