State of the Student Learning Outcome in the Academy
|
|
Over the past month I’ve consulted with both K-12 and higher education leaders in the U.S., Mexico, and the Middle East. Tracking student achievement and the value add provided by an academic program is high on nearly everyone’s priority list. In fact, in a time of shrinking resources this is one area that is still receiving budgetary support.
In his October 26 article Assessment vs. Action on the Inside Higher Ed website, Scott Jaschik summarizes the results of a survey sent to senior academic leaders at 2,809 regionally accredited institutions in the U.S. The survey was commissioned by the National Institute for Learning Outcomes Assessment (NILOA) which is a joint project between the University of Illinois and Indiana University.
Academic leaders and assessment experts should read the Jaschik article and also bookmark the NILOA website which is an excellent collection of resources and current thought surrounding learning outcome management. Conclusions from the NILOA survey were just released in a report titled “More Than You Think, Less Than We Need: Learning Outcomes Assessment in American Higher Education”.
Essentially, the survey found that nearly all U.S. institutions are actively measuring student learning outcomes driven primarily by accrediting body requirements. The gap that remains, however, is to actually use this data to improve student achievement.
Forward thinkers are actively developing a culture of assessment on campus. They’re using assessment data to drive decisions about everything from curriculum and instruction to admission standards and to inform the strategic planning process (Kuh and Ikenberry, Jaschik, 2009).
Over the past year I’ve been fortunate to work with progressive thinkers who are leveraging technology to enhance the outcome management process and to maximize the time that faculty spend providing meaningful feedback and support to students. I look forward to continuing this work and to identifying and publishing best practices in outcome management.
References
Jaschik, S. (2009, October 26). 'Assessment vs. Action. Retrieved November 2, 2009 from Inside Higher Ed, Web site: http://www.insidehighered.com/news/2009/10/26/assess
National Institute for Learning Outcomes Assessment (October 2009). More Than You Think, Less Than We Need: Learning Outcomes Assessment in American Higher Education. Retrieved November 2, 2009 from Web site: http://www.learningoutcomeassessment.org/NILOAsurveyresults09.htm
Brian McKay Epp
Academic Trainer and Consultant
Data
I just got off the phone with a colleague who has lost 35 pounds in 2 months. How did he do it? Data. Well, data mixed with exercise and technology to be more precise. He tried the Nike / iPod experiment and he’s a believer.
This professor of communications and lover of cheese steaks bought a new pair of running shoes a few months back. Then, he bought the Nike sensor system – a small sensor you put in your shoe somehow. This sensor sends information to your iPod during a run. That data tells you (in real time) how you’re doing, but it also allows you to see any trends in your running after you upload the data to the Nike+ website. Apparently he’s run about 340 miles and his average speed has increased by 1 mile per hour. He can tell you how many calories he’s burned and he’s delighted to tell you how many pounds he has lost.
See, data is changing how we live. And data aggregation, data mining, and data analysis are making our lives better as technology gives us more and more ways to use it quickly and easily. For example, my wife was called a few months back about her credit card. Visa thought she might have lost her card. Why? Because she purchased a dress that was 2 sizes too big! Guess what? Her card had been stolen. (No, she had not gained any weight…that would have been awkward!) The credit card company looks for patterns and found something odd in the behavior of the card. So they checked.
Data is everywhere we look today. New cars will tell you how many miles you have driven on a tank of gas and how many more you are likely to get out of that same tank. There is a website where you can upload a sickness in your family. Then, you can look around your city, state, or the entire country to see where other people are sick too. Data might help you avoid the plague!!!
Data is useful and becoming easier and easier to digest. My phone tells me when my flight is late – a handy little feature when you fly 100,000 miles a year. My refrigerator tells me when the filter is no longer doing any good. Heck, even my daughter’s baby monitor tells us when the battery is low. From weather patterns to traffic patterns, data can make our lives tremendously easier.
So why is it so hard to find data for schools? This is especially true with online schools. Shouldn’t you know where your students spend their time in classes? Don’t you think knowing how often you’re B students post vs your D students post to a discussion would be a good piece of information? Does the first day a student checks into class help determine their probability of dropping? If you don’t know the answers to these questions...it’s time to.
One of my favorite tools I’ve ever gotten to work with is a business intelligence tool, created by IBM, that we overlay classes with in our system. This tool allows me and my team to try and predict success, correlate at-risk behaviors to drops, and find benchmarks to hold students accountable to. Did you know that in most online courses a larger class size (30-35) tends to have a better completion rate than classes with less than 30? It’s been proven time and time again through data. (Mind you – data can also beg lots of questions!)
Data mining is becoming easier and easier as technology evolves. Data analysis is becoming more and more automated. It’s time for your school’s programs to join the party! Trends and operational reports are crucial to making accurate predictions and drawing quality conclusions today. Accreditors are soon going to see this power and demand evidence of data-driven decisions for their schools. But before the ‘stick’ of accreditation swats at you, shouldn’t you look to the carrot of quality? Granted, this power can be abused. (My boss loves to look at my completion rates and give me grief as my public speaking class isn’t the highest completed class on campus…it’s public speaking!) But the data is there whether you mine it or not. The information to help you increase retention is sitting there whether or not it’s analyzed.
We study, analyze, and mine data for everything else today. It’s time to get education up to speed, don’t you think? Now if you’ll pardon me…I need to get to a store to buy a sensor. My pants don’t quite fit like they did last year…
Jeff D Borden, M.A.
Senior Director of Teaching & Learning
Actualizing Assessment Accountability
The higher education community has been rumbling for several years about whether and to what extent government will pressure accrediting bodies to figure out how to hold institutions more accountable for demonstrating student achievement and growth. Tuition increases have consistently outpaced inflation for years which has led to public pressure to figure out why and subsequent calls for schools to justify their value to students, parents, and employers.
Legislators responding to public discontent began to increase calls for colleges and universities to actually prove their academic programs were meeting learning outcomes and to demonstrate their value add. The 2006 U.S. Department of Education report A Test of Leadership: Charting the Future of U.S. Higher Education created quite a buzz as the committee’s conclusions started to reach campus leaders. What is yet to be determined is whether it was a temporary blip that stirred up emotions and fears or whether it will truly be an impetus for significant reform.
A February 2007 meeting of the federal accrediting panel is an indication that increased accountability will likely come eventually but it will take time and may continue the U.S. tendency toward voluntary compliance. The most likely scenario is that we will gradually move toward standardization of outcomes beginning with general education which will in turn lead to a growing number of institutions who implement standardized achievement tests.
There are three assessments today that are being administered on campuses and have been approved by the Voluntary System of Accountability, a program that is being piloted by 239 early adopters in the public system under the auspices of the National Association of State Universities and Land Grant Colleges and the American Association of State Colleges and Universities. A similar program is underway for over 700 private, non-profit institutions (U-CAN) sponsored by the National Association of Independent Colleges and Universities and the Transparency by Design initiative is yet another effort that includes a range of private for-profit and non-profit institutions.
Schools that achieve top scores on these assessments will tout their success which will lead to new categories among prominent ranking systems as evidence that students should attend their programs because of their proven track record of achievement. Eventually the accreditors will then push for similar accountability within undergraduate professional majors and before long most institutions will be on-board.
eCollege is prepared to collaborate with institutions in the efficient management of learning outcomes which includes reporting of successes and challenges to stakeholders. Our teams are developing tools that will track outcomes and link them to course content so faculty, department chairs, and deans will have the information they need to assess program and course effectiveness both for curriculum enhancement and for reporting out to accrediting bodies and employers. We will also be able to show students how well they’re doing against the learning outcomes set by their institutions, degree programs and courses. Imagine the power of being able to pinpoint which assignments or test questions are most effectively contributing to student learning.
Brian McKay Epp, M.Ed.
Academic Trainer and Consultant
Categories
Blogroll
Recent Updates
| M | T | W | T | F | S | S |
|---|---|---|---|---|---|---|
| « Aug | ||||||
| 1 | 2 | 3 | 4 | 5 | ||
| 6 | 7 | 8 | 9 | 10 | 11 | 12 |
| 13 | 14 | 15 | 16 | 17 | 18 | 19 |
| 20 | 21 | 22 | 23 | 24 | 25 | 26 |
| 27 | 28 | 29 | 30 | |||