Let’s say you’re an Instructional Designer helping an instructor design a brand new course on engineering project management. Maybe they have a lot of industry and/or teaching experience, but they’ve never taught this particular course before (or maybe have never taught online before), so you’re not quite sure how to get them started. You know the basics of good course design and development, and the instructor probably has some ideas already about textbooks and topics. But you’re both aware of how much you don’t know about this new course.

So what do you do? 

If you were an Instructional Designer at the Office of Digital Learning at OU, you would probably suggest a benchmark survey to the instructor. 

What is a benchmarking survey?

When an instructor teaches a new course (or a college launches a new program), we often do research to see how other institutions run that particular course or program. Usually this is done at the request of instructors and/or program directors, but sometimes it is suggested by the Instructional Designer.

At ODL, we do two types of benchmarks: course surveys and program surveys.

Course benchmarks provide information for instructors teaching a course or material for the first time. They can also offer guidance on how instructors can revise existing courses or update courses that are changing delivery format or length. Course benchmarks provide Instructional Designers with additional context that helps them prepare for design meetings with instructors and program directors.

Program benchmarks on the other hand identify opportunities for programs to provide niche offerings or innovations that distinguish them from their competitors. They help program directors create course lists, course syllabi, and materials for program accreditation applications.

Why should you do a benchmark?

We have found both course and program benchmarks to be really useful for instructors and program directors. In order to make informed decisions in the course design process, instructors and program directors need as much information as possible. Benchmarks give quality context and examples of what other universities are doing, in order to help faculty develop courses and programs that benefit students.

The research is used to ensure that the format and content of our programs and courses are in line with what other institutions offer. The goal is not to match other courses exactly but to make sure we are staying committed to established best practices. As Instructional Designers, we’re not using benchmarks to say that an instructor’s course should look like other courses. In fact, the benchmark often helps the instructor determine what niche offerings they can offer that other institutions aren’t. At the end of the benchmark, the instructor still has the freedom to design the course that fits their style, the college’s requirements, and the students’ needs; but now they have a wealth of extra information from other universities to support that process. 

There are loads of other specific reasons why an Instructional Designer might do a benchmark for a course or program:

• To figure out how to cut down a 16 week course to 8 weeks 
• To help put together materials for accreditation
• To help transition an in-person course to online
• To update a course that has been taught for a while

What is the benchmarking process?

Our course development process typically involves seven phases:

1. Discover
2. Design & Develop
3. Plan
4. Build
5. Review & Launch
6. Manage
7. Evaluate

We typically utilize a benchmark during the second phase: Design & Develop, which usually starts after the initial kick-off meeting and alongside the early design documents. A benchmark is most useful early in the development process, where it can help inform the planning and building phases. The goal is to finish the benchmark before the instructor creates their first prototype module and films their course overview video. 

But in order to do a benchmark, we need certain basic information from the instructor and/or program director. In general, the following information is helpful before starting the benchmark:

• Background information about the course/program
• Past syllabi (if available)
• Purpose of survey and areas of focus
• Example keywords to search
• Institutions to prioritize (if possible)
• Time allotted for the task

After all the initial information is collected from the instructor or program director, an Instructional Designer or Course Developer can begin working on the benchmark. In general, benchmarking follows four basic steps:

1. Collect as much information as you can (via course syllabi or program pages) from as many universities as you can 
2. Organize that data by university surveyed and categories (We use a Google Sheet.)
3. Analyze data for trends and findings
4. Create report to send to the instructor or program director

The specifics of how to complete each step depends on whether you are benchmarking for a new course or a new program, though. 

How do we do a course benchmark?

The first step in a course benchmark is to search for similar courses at other universities. We are looking for courses with similar titles, level, delivery methods, length, and topics. We are also looking for courses that have made their syllabi available online somewhere. Finding appropriate courses is often the most difficult part of the process, but it is essential to collecting quality data.

Once we have compiled a list of potential courses, we go through them, taking note of the basic information (course number, title, description, instructor, URL, etc.) and then start compiling information about how the course is designed. In a general benchmark, we are most of the time looking for the following:

• Learning objectives
• Course topics
• Assessment strategy
• Grade breakdown
• Module structure
• Course materials

Once we narrow down our course list to the most relevant courses and compile all the information, we will take some time to analyze the data for trends, examples, and takeaways. Our goal is to condense a handful of best practices for that course topic or program to help the instructor plan the course.

Once the information is analyzed, we will start writing a report for the instructor to view. This report will go over our methodology, any limitations, and our general findings. We try to make the report quickly scannable by including short summaries and multiple graphs and tables. But we also include more detailed information in the appendix and link to our full notes and the other course syllabi as well. 

How do we do a program benchmark?

For a program benchmark, we start by searching for comparable programs at other universities. We’ll then look at the corresponding program or department pages on other universities’ websites to see if the program is in fact similar to what we’re looking for. Once we compile a list of potential programs, we go through a similar process to course benchmarking – taking basic notes about each program and compiling data about each program, which usually includes the following:

• Program outcomes
• Curriculum topics
• Evaluation methods
• Course sequence
• Unique program features
• Program tracks
• Program accreditation
• Target audience

Once we have compiled all the information we can, we will analyze that data and turn it into a readable report for the program directors to review. The program benchmarking report is similar to the course benchmarking report. It includes methodology, limitations, and summaries of key areas with corresponding tables and graphs. It will also include a more detailed appendix and links to the other programs. 

Empower your instructors with benchmarks.

We have found that benchmarks are a great resource to give instructors who are designing a new course or changing an existing course. They’re also very useful for program directors trying to organize a new academic program. Even if you don’t have the time to do a full, thorough benchmark, though, it’s always worth looking at what other programs and instructors are doing. 

Completing benchmark reports is a great way for Instructional Designers to support instructors and program directors in the design process. By providing them with information from other universities, they will be more empowered to create great courses and programs to benefit instructors, students, and the university.