Using Programmed Instruction

Helen Lester Plants

Department of Theoretical & Applied Mechanics, West Virginia University

April 13, 1967

Most of us are aware that we are working in a difficult era in engineering education. The information explosion demands that we teach more material and the population explosion demands that we teach more students, but the number of credit hours in the curriculum and the number of working hours in the day remains uncharged. As a result we all teach more less thoroughly, and give more students less individual attention than we did ten years ego. We all worry about it, more or less, but nothing much gets done about it.

Perhaps the solution for our problem is to realize that our old educational methods, excellent though them may have been, are no longer adequate, and to devise and investigate new methods to augment or replace them. One such new idea in education is programmed instruction.

Programmed instruction has been around long enough that most people have heard a bit about it, but have never actually come to grips with producing it or using it. At West Virginia University we have done both. We have found programmed instruction to be effective - sometimes amazingly so - and we have found that it has given us new insight into the problems and techniques of education.

Programmed Learning : An Evolving Definition

Before we can do much talking about the use of programmed learning materials it would be well to-establish firmly what we are talking about. It is fairly easy to find definitions of programmed learning in the literature - too easy, in fact, since one finds definitions rather than a definition. As a result, two people discussing programmed learning are often talking about slightly different things. In fact. the very definition of programmed instruction seems to be evolving as our knowledge of programming increases.

Apparently the first generally accepted notion of programmed learning was that is was some sort of educational scheme making use of a set of rules, and even gimmicks. Consequently, anything that was written in a certain format - usually question and answer was automatically tagged programmed material. Some of it was programmed, and some wasn’t.

The basis of the early programs was behavioral psychology - actual laboratory experiments to see whether teaching affected the behavior of subjects in a way which could be objectively measured. The "rules" and format of early programs grew out of such tests as the most effective methods for affecting behavior were identified. Thus we began to hear about immediate confirmation, active responses, short learning episodes and all the rest. This was also the era of much hardware in the form of teaching machines and very peculiar looking books, and the people said, and to a certain extent still say, "So this is programmed instruction."

It is true that most programmed instruction has some of the previously mentioned characteristics. However. the point which determines whether or not a piece of material is a program is whether or not is whether it sets out to change the learner's behavior in a specific objectively measurable fashion. If it tries to do that, it is a program. If it succeeds, it is a good program. Its format is irrelevant.

Another fairly common notion of programming is that it is a method of manipulating the student's behavior so that his energies are channeled into the correct paths. This view of programming sees it as a long series of actions on the part of the student which are closely supervised by the programmer. This supervision may be quite sophisticated, taking advantage of the newest in audio-visual aids and latest thing in computers as the student is led along an educational path tailored to his exact needs. The display of cleverness in supervision can be quite dazzling and impressive so that many people think of programming as a sort of remote control teaching process.

Certainly some programs make use of just such techniques, but it is not the techniques that make them programs. Unless it can be proven that specific objectively measurable changes in behavior will take place as a result of the course of action, it is not a program.

Parenthetically, an example of something that looks like a program but isn't is the famous dialogue between Socrates and the slave boy which results in the boy having proved the Pythagorean theorem. As you probably remembers, Socrates, by carefully constructed questions, elicits each step of the proof from the boy. In doing so Socrates certainly makes use of the latest ideas about eliciting the desired response and reinforcing it. He controls the subject's responses masterfully and there are many who would say, "Aha, the first recorded instance of programmed instruction." Unfortunately, it is no such thing. Socrates omitted a post-test. To have produced a program he would have had to have shown that at the end of the dialogue the boy was able to do something, i.e. produce an unprompted proof of the Pythagorean theorem - that he had not been able to do before. Socrates, of course, would have been horrified at the suggestion that he was teaching the boy anything, and indeed he wasn't. Looking at the dialogue from the point of view of the classroom teacher most of us will agree that the slave boy would have flunked the post-test flat. Since no learning was demonstrated the dialogue was not a program.

So the question remains, "What is programmed instruction?" The answer that currently seems best and that we find most useful is that programmed instruction is instruction designed to achieve specific scholastic goals and proven by test to do so. The goals must be specified in such a matter that it can be proven that they have been attained.

Objectives and Post-Tests

The specification of the goals of programmed instruction involves two stages - definition of objectives and design of post-tests. Before going further it would be well to take a closer look at objectives and post-tests.

The objectives of a program are, rather obviously, what the programmer hopes to teach by the program. Objectives, however, must be clearly defined in terms of observable actions on the part of the student. We cannot say we want the student to understand a certain body of material. We must say that we wish him to demonstrate his understanding in a specific fashion. We could require that he work problems involving specific tasks; we could require him to write definitions or proofs; we could ask him to fill in blanks; or we could ask him to do all at these things and more. The important thing is that the objectives be stated in terns of actual measurable actions on the part of the student and the: these objectives be such that all observers will be in agreement as to whether or not they were met.

An example of an unsatisfactory objective might be "The student will know the alphabet." How do we know he knows it? If he doesn't choose to demonstrate his knowledge we can never determine what he knows. Furthermore you may feel that your child knows his alphabet because he can recite it; but his teacher may feel he does not know it because he writes several letters backwards. Thus, if we are not careful to spell out our objectives in precise behavioral terms, different observers may not agree as to whether or not the objectives are met.

Satisfactory objectives for the same bit of instruction could be "The child will recite the alphabet." or "The child will properly identity all letters of the alphabet when they are shown him." or "The child will, upon hearing a letter named, write the script character for it." Any of these might be construed as "knowing the alphabet" although they require very different skills. It is evident to any observer whether or not any of these has been met.

Properly defined behavioral objectives are half the battle in any piece of programmed instruction.

After the objectives come the post-tests. Post-tests are simply the instruments by which the programmer measures the degree to which the program has succeeded in meeting the objectives. Post-tests can be very simple or very complex but to be valid they must correspond to the objectives of the program.

There are two types of tests which are called post-tests. This is unfortunate because they are quite different.

The first type might be called a design post-test and is very detailed. It tests the student on every step of a learning sequence. The simple ideas which are included in the final objective are as carefully examined as are the final objectiveitself, with the result that if the student fails to learn it is possible to see exactly where the program was inadequate. If one gave a post-test consisting of complex problems and the student solved them all, he could be assumed to have mastered all the simple ideas as well as the complexities. But if he missed all the problems it would be impossible to tell where his trouble lay. The elaborate post-test used in the design of a. program is a diagnostic test to determine where the program does not teach. The design post-test tests the program.

After a program has been proven by reiterated testing with a design post- test, a shorter post-test becomes practical. It is now possible to give a few critical. items and by the student's satisfactory performance on these items to inter that he is capable of all the work that preceded them. These are the post tests that are usually included with published programs and they differ little from ordinary tests. The program's capacity to teach has already been proven. The point at issue is whether or not the individual student has learned. The final post-test tests the student.

The final post-test can lead to misleading conclusions if the type of error made is not given careful consideration. Sometimes programs are better than the post-tests make them look at first.

An example of this sort of thing occurred on the last test given our programmed dynamics sections. The subject matter of the test was force, mass and acceleration relationships for rigid bodies. The results of the test were disappointing The grades were not really bad hut were certainly not as good as we had come to expect.

The individual units had post-tested reasonably well. An error analysis of the tests showed us what hat happened. Almost every student in the class was either unable to draw correct free-bodies or to calculate moments of inertia for geometric bodies. Some could do neither. Of course these topics had been assumed prerequisite knowledge since they had been taught in statics. The errors stemming from the actual subject matter of the programs had not been very numerous at all.

The Design of Instruction

Programmed instruction is designed instruction with the word "design" used in the engineering sense. The programmer attacks the problem of educational design exactly as he would a problem in machine design.

First the engineering designer defines as carefully as ha can the function of the thing which he is designing. What will it do? Next he sets up a set of performance specifications. It must do exactly this under exactly these conditions. Computed stress must be less than this, deformations must not exceed that.

The educational designer does the same thing. First he decides exactly what his program will teach. He determines exactly what he wants the student to be able to do when he finishes the program. He defines this terminal behavior as carefully as he can and calls it his objectives. In determining his objectives he makes certain that he sets them up in such a way that their accomplishment can be demonstrated by actual test. Next he turns his attention to the construction of his post-tests. These are the specifications he is going to meet. He is going to develop a program such that when a student has completed it he will be able to pass the post-tests. Nothing less can be considered a satisfactory design.

When the engineering designer has decided upon function, his next step is to actually design his device. The next step for the educational designer is to write his program. As the engineering designer calls upon his knowledge of mathematics and physical science and incorporates previously proven elements into his design, so the educational designer calls upon his knowledge of psychological principles and makes use of teaching techniques of proven efficiency. It is at this point that both designs begin to take on familiar shapes. The car begins to look like other cars and the program begins to take on the format we associate with programs.

The engineer's next step may be to build and test a prototype. If it does not fulfill its function and meet his specifications, he will redesign and test again and redesign and test again until it does meet them. He will examine each part of his design - each bearing and bushing - until he is certain that each component is functioning properly.

The programmer does exactly the same thing. He tests his design by trying it on a student and then post-testing the student. If the student fails the post. test so does the program. It must be rewritten and rewritten again until each frame produces the desired response and until the post-test is passed.

When the engineer has achieved a machine that works well under laboratory conditions he its ready for a field test. If the results are favorable it is, at last, time to go into production.

The last step for the programmer is the same. He has produced an educational design that meets specifications when used on individual students or on a class or two. Now he must put it into a large scale class-testing phase, preferably in several schools. If it still meets specifications, if the students are still passing the post-tests, then it is ready for publications.

As engineers we are all aware that such a design procedure works for engineering problems. It works just as well for educational problems. A program that has been properly designed, tested, and validated will teach what it sets out to teach. Students will pass the post-tests and the objectives will be met. You can rely on it.

Using Programmed Instruction as a Supplement

For the last five years we have been using programs in undergraduate mechanics courses at West Virginia University. We have found out that they do work and we are finding out how to use them to beat advantage.

Our first attempt at programmed instruction was a branching program on writing shear and moment equations for beams. In retrospect it was a pretty poor program It was given to the students to use for home study in place of homework problems. Considering that it consisted of some seventy pages, the instructor was somewhat reluctant to face the class on its next meeting. However, instead of being disgruntled the students were delighted. So was the instructor! The students were actually asking questions about the material customarily covered in the third lecture on the subject. They already knew the first two lectures!

From that day on, the pattern for adjunct programs was set. We had discovered a way to teach drill topics efficiently. We no longer needed to use class time for repetitious work designed to build skills in problem solving. Neither did we have to tunnel through mountains of homework to determine how well the skills were developing. We could simply hand out a program, tell the students when it would be due and give a ten minute quiz on that date. The performance on the quiz would be high and the students would know the material well enough to go on.

Figures 1A and 1B show the results of post-tests on two adjunct programs. The actual tests given were similar to but not identical with those shown on the figures. Since three different sections were involved it was not practical to use identical tests. However, we saw practically no variation in grades from section to section.

Figure 1A. Post-Test Scores on Mohr's Circle for Stress. Three T-Th sections in 1964-65

Figure 1B. Post-Test Scores on Mohr's Circle for Strain. Three T-Th sections in 1964-65

We immediately identified the drill topics in Mechanics of Materials, programmed them, and applied the time that we thus saved on other work. We found that for the first time in several years we were actually getting through all the material in the course outline.

We also made our first discovery about using programs. It was simple. You post-test. If you omit the quiz over the unit on a specified area, the students will either put off doing the unit indefinitely or just plain not do it. The quiz doesn't have to be hard, it doesn't have to be long, it just has to be inevitable. Furthermore it should be taken from the exact material covered in the unit and it should count on the student’s grade. (We usually count it as a homework problem.) This is necessary to make the student feel that doing the programmed work is important and a real part of the course, not just something thrown in as an after thought.

The programs that were developed and used in the Mechanics of Materials course were adjunct programs. That is, they were designed to be used as supplements to a regular textbook but not to any particular text. A good program of this nature should fit in and be helpful when used with any standard text in the area. Neither does the use of such programs require any special lecture technique. To get the greatest advantage from the programs, however, the lecturer must learn to trust them, and not feel that he must cover the programmed material in his lectures.

Programming an Entire Course

Nothing succeeds like success and before long students were asking for more programs, particularly in Dynamics. At about that same time we became a part of the ASEE programmed learning project whose purpose was to produce engineering programs. Consequently a start was made on programming dynamics.

The same approach was tried that we had been using in mechanics of materials. Certain topics were selected and programs were written and tested. This time we employed a linear program with occasional branching loops. Once again, the students were delighted but this time the programmer was not. Having learned a bit more about interpreting post-tests, it was impossible to avoid a very painful conclusion. The reason for most of the errors on the post-test lay not in the programs but in the material the student was supposed to know before he began the unit. The only sensible thing to do was to start at the very beginning and do the whole thing.

Consequently in October 1965 we began to look toward an entirely programmed dynamics course. In the spring semester of last year we had programmed enough to teach the first seven weeks. Last fall the programs ran into the twelfth week. This semester, by making use of some material from Dr. Clyde Work of Michigan Technical University, we were able to offer a completely programmed course.

This is how we do it. At the beginning of the term the programs, twenty of them this term, are checked out to the students. Each student also receives a schedule of post-tests that tells him when each unit will be post-tested. On the appointed day the post-test is given, the papers are collected, and the post-test discussed. The post-test grades are posted as quickly as possible. At appropriate intervals, hour tests are given. The hour tests aim at determining whether or not the students are integrating the ideas from the individual units into a coherent whole. Any student whose average at the end of the term is 75 may take a C without final examination. If his average is 85 he may have a B without a final. All students with averages below 75 must take the final and any student who thinks he can get a better grade by taking the final may do so. It is not possible to make an A without taking the final.

As it stands it is a pretty uneven affair. Some programs have been class tested and revised three times and are getting pretty good. The newest are being used for the first time in our classes right now. (Work, however, has used the latest units in his classes and found them fairly effective so that all material currently in use at WVU has been class tested somewhere prior to its use in this term's class.

We have discovered teaching with programs is very different from teaching a course with adjunct programs. It is exceedingly effective, but it changes the teacher’s role completely.

We had our first indication of its effectiveness rather early in the game. In selecting the material to program the writer at first chose not to cover those topics which had seemed to be rather effectively covered in lecture in the past. In other words, she taught a series of programs pieced together by what she considered her best lectures over rather easy topics. In due time there was a test. To the teacher's embarrassments, practically all points lost on the test were on material from the lectures. By the next term the gaps were filled with programs.

At about this time it became possible to make some comparisons of programmed and unprogrammed classes. Rather than attempting to use a current class as a control, the current classes were compared to past dynamics classes. The figures show some of our results. By going through the file of old tests we were able to reuse a test of some years before and compare the results. Considering the large number of tests we had to choose from, we considered the likelihood of A student having actually studied the test he took to be quite small. The tests were probably slanted somewhat against the current students since they used some unfamiliar terms of phrase and symbol that were the legacy of wherever text we were using the first time the test was given.

Figure 2. Comparison of scores on matched tests.

Figure 3. Comparison of scores on matched tests.

Since our first all-programmed class is still in progress it is not possible to give comparative data on it. However, the next figure shows a comparison of class averages on hour examinations between the partially programmed classes and the average of all unprogrammed sections taught by the writer since we began teaching vector dynamics. Each class took four hour exams during the semester. An interesting thing to note is what happened to the classes when the programs ran out.


Figure 4. Average Examination Scores of Programmed and Unprogrammed Classes

In addition to the sort of data shown in the figures we learned a lot of things that cannot be reduced to numbers.

We have found that programs do not effect any saving in the student's time. In fact he probably spends a bit more time on his dynamics than he would otherwise. However, the time is spent more effectively since he does not make numerous unsuccessful tries at the same problem as he probably would on the ordinary lecture homework schedule. Since he is constantly reinforced by getting right answers he puts in the time fairly willingly.

The programs seem to have done most for the worst students. The students with the lower grade point averages often report very long times spent on the units. However, by being able to go slowly, they do learn the material as is evidenced by the post-tests. Apparently if a student has sufficient intelligence to have passed the pre-requisites and is willing to invest the time he needs to work through all the programs he can be assured of passing dynamics. It appears that the poor student is not so much incapable of learning as he is incapable of learning fast enough to keep up. The programs fix that if he wants them to.

One early discovery was that given a textbook and programs, students will generally ignore the textbook. Our early programs in dynamics were considered as adjunct programs to be used in addition to a textbook. Quite soon it was apparent that the students were not using the text at all. If they were taxed with this they readily admitted it, saying they didn't have time to do the text and the programs. Since the programs were designed to take two to three hours per assignment this sounds quite sensible except when one realizes that the reading assignment took only ten to fifteen minutes. We had expected the students to use the book as a concise summary of the programmed material but we couldn't get them to do so.

Another thing we observed was a sort of inertia effect. Most students seem to have a deep distrust of anything new and rather resent the first few programs, feeling perhaps, that they are being used as guinea pigs. After working through a couple of programs their suspicions are allayed and almost all become mildly enthusiastic about them. A few get to be real fans. Comments that have gotten back to us range from, "They are awfully long and tedious to work through but it you do, you can't help learning it." to "The thing I can't get over is how much I actually know."

There are a few, however, who are never converted but remain solidly against programs to the last. One investigator at another school has told us that he has found such holdouts to be either at the top or the bottom of the class. That makes good sense and could be explained easily. The only trouble is that in our experience the "holdouts'' have been rather firmly in the middle of our C range. Our feeling is that it is probably simply a matter of individuality. These hold-outs run about one to a section and since they do presentable work despite their feelings in the matter, there doesn't appear to be much reason to worry about them. After all there is probably one student in any section who can' t stand his particular teacher anyway.

Shortly after our discovery that the programs were replacing the textbook we discovered they were also replacing the lecturer. That was unnerving, to say the least. The old lecture starter, "Any questions?" drew a blank. No one ever seemed to have any. It the lecturer insisted on explaining anyway, the class attitude was one of polite patience; they already knew the material. Post-testing bore them out. They did know it.

So what does the lecturer do in a programmed class? The answers very.

Dr. Work, in his programmed dynamics course at Michigan Technological University, has one answer. He simply meets his programmed classes to give tests and return them. Otherwise his students are on their own. He has not yet reported his results in detail but was finding them quite satisfactory when last heard from.

At West Virginia University our approach has been a bit different. After taking care of the details of post-testing, the instructor simply goes a bit beyond the material in the programs, offering different viewpoints and applications, demonsstraying more difficult topics and so forth, There is sort of a gentlemen’s agreement that this is enrichment material and not to be tested on since it is outside the objectives of the course as set down in the programs. This sort of class has resulted in some rather spirited discussions and seems to be of considerable interest to both the students and teacher.

This format seems to make for excellent student-teacher relationships. The writer, for instance, finds that she now knows almost all her students by name without particularly trying to do so. Also many more students take advantage of her office hours than in past times. All in all, we seem to do a better job of teaching because we know the minimum necessary information has already been taught. That frees both teacher and students to consider those things which interest them most.

Next fall we plan to try a new way to teach with programs. Rather than the usual three-times-a-week schedule, the class will meet weekly for a three hour session. During each session we will post-test the assigned programs, have a discussion period and spend some time on a simple dynamics experiment or two. We hope to save some of our starting and stopping time and also to foster a healthy, hands-one experimental attitude in our students. We feel that this may result in more effective use of class time than that we are now making.

(Note: This experiment was never conducted.)

We have found out a lot more about the effectiveness and ineffectiveness of various teaching techniques than we can possibly discuss today but all in all we have been greatly strengthened in our notions of the effectiveness of programmed instruction.

Programs for You?

Perhaps some readers would like to try a bit of programmed instruction on their own classes and are wondering how to get started. The best course would probably be to start just as West Virginia University did with an adjunct program or two. Find an area in the course where it seems that a program would help and try one out. It will probably save a couple of lectures so that the teacher will find him self looking around for more programs, just as we did.

In selecting a program, the starting point for the teacher is exactly the same as for the would-be programmer - he must define his objectives. He must decide what he really wants his students to be able to do when they finish his course and what he considers satisfactory proof of that ability. If any of his objectives fall in the concrete area of knowledge or information, rather than the more abstract realm of wisdom, such objectives are fit topics for programmed learning.

Having gotten his own personal objectives firmly in hand, the next thing for the teacher to do is to hunt for a programmer who has exactly the same objectives. He won't find one. Just as with textbooks, the only program that exactly suits any teacher will be the one he writes himself. But, just as with textbooks, he may find one that fits his needs well enough.

How does one tell if a program is what he needs? Look at the post-tests the author provides They will probably be composed of the terminal questions from his design post-tests, although he may actually include his design tests. At any rate, if his post-tests look about like the test questions you usually ask, then you can be pretty sure his objectives coincide quite well with yours.

How can one be sure a given program will teach? One can't but there are some things to look for. First of course; the very fact that the author suggests post-tests implies a certain confidence on his part in the ability of the students to perform satisfactorily on them. Better yet, the author may actually publish validation data with his programs. Or he may not, since it is not always published even when available. If it is not published as a part of the program, the overall results of validation tests may be described in the foreword or preface of a published program. If he can find no evidence of. an effort at validating a program the teacher’s attitude should be quite skeptical, but if its objectives coincide well with his, it is probably worth a limited try. It has been the experience of several workers that even rather bad programs often teach fairly well and are welcomed by the students.

The one thing this writer would not suggest doing in deciding about a program is to try to read through it. Programs are not meant to be read. They are meant to be done by naive subjects. That means that they progress by such baby steps that they are pure agony to read if you already know the material. It seems that it is almost a truism that in order to be good a program must appear trivial - to an expert in the field, that is.

Most of the so-called trivial frames will have been inserted, however, to help the student fill in details which tests showed were not nearly so evident as the programmer had previously thought them. In one of the writer's programs, one particular frame seemed the obvious out-growth of the one before it - until the first trial run. The subject failed to see the relation. A couple of frames were added to clarify the relation. The next subject failed to see the relation. A couple of frames were added and it was tried again. Again the next subject failed. This was kept up until it had been tried five times and nine frames had been added to clarify the relation that had seemed obvious to the teacher. Now it was obvious to the student and painfully so to the teacher.

One may look at a few frames of a program to get the feel of it, but he should never try to read it through. He'll never make it and he'll never use a program. He should rely instead on what can be found out about the objectives of the author and how well he was able to accomplish them.

If any one does decide to try out a program And wants to know how well it taught, let him look carefully at the post-tests his students write He should note the exact kind of errors his students make and see whether they indicate a poor knowledge of the programmed material or of the material pre-requisite to the program. He may find himself programming the pre-requisites.

Available Programs in Mechanics

This is a particularly good time to get started with programs since the ASEE is, at this moment, actively soliciting volunteers to aid in the validation of the programs produced as a part of its Programmed Learning Project. Details of the project and information on how to obtain programs may be obtained from the February 1967 issue of the ASEE Journal.

In the specific area of mechanics there are about thirty different programmed units in various areas of dynamics. Dr. Work has done eight to ten of these and the writer has done the remainder. There is some overlap as each author is aiming at a fully programmed dynamics course of his own. Each topic is developed in such a way that you may use just one or several as you choose.

All these programs are unpublished but are expected to be available through the ASEE There its also at least one published program covering some topics from strength of materials. Several publishers have plans for programs in statics.


The claims made for programmed learning are many. In our experience they are mostly true.

Students, by using programmed materials have been able to perform very well in our mechanics courses. Each is able to work at his own pace, review as much as is needful and be confident of his results. If he is willing to invest as much time as the programs require he can be virtually assured of a passing mark.

For the teachers the advantages have been a considerable reduction in routine work and a certainty of a reasonable performance by the students. It has freed the faculty to devote class time to those non routine elements of teaching which characterize education at its most interesting and best.

Those of us who have used it, both as students and teachers, at West Virginia are convinced that programmed instruction is the educational design of the future!