General Engineering Introduction/ASEE Paper/Internal Grading
The Internal Grading Problem
Many different Introduction to Engineering grading systems have been tried including problem/solution, all or nothing team, milestone submissions, time spent, and writing volume. All have problems fitting into any kind of engineering narrative. The “doing things first narrative” fits a portfolio context. Individual portfolios graded by instructors is promoted below. Team documentation assessed by outside experts is more appropriate for program/course assessment.
Portfolios
editK-12 and Post Secondary students from around the country working on individual and team projects have been using the Innovation Portal to document their design projects. https://innovationportal.org/stats The portfolio template is organized around the Engineering Design Process Portfolio Scoring Rubric (EDPPSR) that is being developed by groups of engineering design educators from around the country under a formal research effort and lead by Researchers from the University of Maryland.
Capturing the design process in a way that can be compared , shared and that can help bridge the gap between K-12 and post-secondary course work is the goal of all of those who have come together to build the EDPPSR and the Innovation Portal. the Innovation Portal is free and open for use by all teaches and students. The rubric, scored and annotated examples, and tutorials can all be accessed on the Innovation Portal site.. https://innovationportal.org/
The Project Lead The Way team has designed, built and continues to improve the Innovation Portal for all students because we are committed to supporting, promoting, and working to gain recognition for student works that incorporate the application and integration of knowledge and critical thinking skills associated with original problem solving and design projects. The Innovation Portal is an excellent vehicle for helping to achieve that goal.
Individual students have been building individual portfolios in the PLTW EDD class. The EDPPSR (Engineering Design Process Portfolio Scoring Rubric) is being used to encourage Engineering schools, the College Board and ABET to increase:
- admissions into other project-based programs;
- admissions into post-secondary studies;
- career pathway recognition; and
- advanced Placement or dual-credit into more rigorous academic courses.
The rubric doesn't measure teamwork documentation, commitment, transparency, contribution, integrity, or persistence. The rubric has tried to create an information control point by establishing a web site to upload portfolios. The portfolio is forced into a one size fits all mold so that an “efficient” assessment process involving outside experts (College Board) can be created. Development has been dominated by technology arts K-12 personnel, not STEM. It doesn't separate individual student grading (feedback on daily/weekly basis that forms the basis of gradual improvement) from program or curriculum assessment (goals are all individual, portfolio is individual). The rubric depends heavily upon subjectivity. EDPPSR hopes that there is a statistically significant, (but as yet undiscovered) “ideal portfolio” vision within the expert engineering portfolio evaluators. The positives of EDPPSR are that it:
- focuses on documentation;
- raises assessment questions;
- is general, not focusing on a particular technology.
Team versus Individual
editThe words “teams and projects” lead students to think of sports teams, winning and loosing. They are surprised when there is an individual grading component in a “project” class. The first step in grading an “introduction to engineering class” has been to create a mechanism to separate “We” and “I.”
Introduction to engineering students want to socialize. “We” will happen with no effort by the instructor. The focus has to be on the individual. Engineering problems need to be separated from personal, “life gets in the way” problems. Ignorance needs to be turned into a problem solving asset. Technician troubleshooting needs to be separated from engineering design. There are lots of “I” issues that start with individuals defining and solving small engineering problems themselves. Open ended projects are merely opportunities to explore small engineering problems.
Atoms versus Documentation
editStudents learn through hands on experiences. Given enough stuff, students will begin playing. If only the final product is assessed, then goal of moving students through playing, doing things first, design and problem solving is not addressed. Students can and will spend enormous amounts of time .. playing .. to get something to work if this is all that is rewarded. The goal of the grading has to be encouraging students to stop and design; to stop and define engineering problems.
There are two ways to get students to stop. The first is to force students to write before, during after they do something of their choice. This improves handwriting, forces carrying engineering notebooks around and captures level of chaotic detail and inspiration that is missed in after the fact, summary writing. Currently this is being done through the GoingToDo, Doing and Analysis triplet. This evolved from the science triplet of Hypothesis, Procedure/Testing, Conclusion.
The challenge is to get students to Analyze. The Reflection of EDPPSR has too much in common with service learning reflection (personal life reflection) and not enough in common with testing, analysis, and conclusions. Analysis is described as picking Socratic questions and answering:
- what went right/wrong
- why something went wrong
- what is the error that is built into the number
- if had more time would have done .......
- a better tool for this job would have been ........
- if could do over again, would have ........
- how this could be done more accurately ......
- if could automate, would improve data by ........
- this was expected, and this was not expected
Three points are given per completed project triplet of "GoingToDo, Doing and Analysis". One point is given if there is no Analysis. Students generate 3 of these triplets per hour on average. This is called project work in the notebook grading system. It leads to design.
Six points are given per completed problem triplet of “problem, possible solutions and testing” that can get 6 points. This quickly raises the issue of “What is an engineering problem.” “I can not pop all the balloons in the Pogo.com game Poppit?” is not an engineering problem. "Can all the balloons in each game possibly be popped?" is. Single solutions are not rewarded. Students must brainstorm possible solutions. Testing is determined by context and has to exist. Working through these issues matures student engineers.
Time versus Accomplishment
editNotebook writing gives a clear indication of time spent. Electronic documentation emphasizes accomplishment. Both are needed. Accomplishment starts off personal and passes from teammate to teammate like a hot potato, each adding something personal. To capture this process, four levels of electronic documentation are needed: weekly personal, weekly team, team summary, commons contribution.
Weekly personal documentation is graded with push points (accomplishment points) that range from 0 to 100. Weekly team documentation is graded 40-80 points, but only if a short presentation is made. Team Summary documentation is motivated by withholding project points from the notebook and push points from individual grading until the Team Summary is done. PLTW promotes the inventing engineer rather than the "do it first" engineer. A Commons Contribution fits into the service motive that forms the root of engineering ethics much more than misleading expectations of invention wealth. Help equate Open Ended Projects with “Open Source” engineering!