a blog about things that I've been thinking hard about

An Internet-Based Examination System

6 February, 2006
internet-based examinations

The Internet is great for those who wish to educate themselves informally and cheaply.

But what if you want to achieve a formal qualification?

This article describes an architecture for an Internet-based computerised examination system.


Formal vs Informal Education

Some people like to learn in a classroom. They are happy to sit there and have the teacher tell them what to learn, and when they get stuck, they are happy that there is someone there to help them, and they enjoy the social environment.

For many people, "education" is a synonym for "formal education", and if you tell them that your education consists of reading and learning by doing, they won't take you seriously. "Yes, but are you enrolled in any courses?"

One advantage of formal education, for those who like it, is the structured learning environment. The other advantage is that it comes with a system of qualifications. That is, you take the course, you do assignments, projects and perhaps examinations, and the results from these determine whether or not you "pass".

If on the other hand, you prefer to educate yourself informally, but you want something to show people that you have successfully educated yourself in a particular area, then you need some means of assessment which is disconnected from your choice of education process.

The Cheapness of Informal Education on the Internet

Self-education requires access to learning resources. Traditionally, the two major sources of self-learning material were books and real-life experience. And if you wanted to include interaction with other people in your educational program, you would have to find other students interested in informally studying the same topic, and if you were lucky you might find some expert willing to informally assist you in learning about the topic.

Modern technology has added a new ingredient to this mix: the Internet. The Internet can be regarded as a cheaper and improved version of the four self-education resources I just mentioned:

A big feature of the Internet is simply how cheap it is. My current Internet plan costs about the same as a book for one month's access, but in one month I am free to download the equivalent of thousands of books.

The cheapness factor is potentially even more important to people who come from poorer countries, who may simply be unable to afford entry to formal education in anything but the most basic and popular of subjects, but who may have ready access to Internet-connected computers for as little as US$0.30 per hour.

A Theory of Examinations

The traditional approach to measuring a person's level of knowledge in a topic has been the examination. These days there is often more emphasis on "internal" assessments, which may consist of assignments and projects given out by the teacher and then marked or assessed by the same teacher. However examinations remain a popular component of many assessments. Examinations have the advantages of:

The Internet creates opportunities for making examinations both more reliable and cheaper than they are already. To see why, we need to look at what the components of an examination process are. Ignoring the efforts of the examinees, there are three major components of effort required to administer an examination:

  1. Create the examination, and keep the contents secure until the examination starts.
  2. Supervise the examination – supervise the entry of students into the examination environment, identify them if necessary, ensure that they do not have prohibited materials or communication devices, make sure they do not communicate with each other, and collect the examination papers at the end of the examination period.
  3. Mark the examinations.

Using computer technology, it should be possible to streamline these three phases, and also disconnect them from each other.

A Computerised Network Based Examination System

The first step in stream-lining the examination system is to computerise everything. The examinations are defined as computer files (ideally in some standardised format). The examinations are taken by the students with each student on one computer. The answers entered by the student are then retained on the computer, later processed by computer, and finally distributed by computer.

We can consider to what degree creation, supervision and marking can have their costs reduced to a minimum:


An examination result is only useful if it can be trusted. If a potential employer judges a potential employee on the basis of an examination result, the employer needs to trust the creation process, the supervision process and the marking process.

Where these are separated from each other, the examination result must explicitly show the parties responsible for each component of the process, i.e. who created the examination, who supervised it and who marked it (or, if it was marked automatically, what algorithm was used to mark it). Anyone judging the reliability of a result must take into account all identified responsible parties.

Business Opportunity

The major business opportunity for a streamlined computer-based examination system is for the provider of the supervision component of the examination process.

The nature of the supervision service effectively determines the framework for the other processes, and also for the distribution of final results.

The services provided will fall into four main categories.

Examination Hosting Services

An examination service will provide the ability for examination creators to create examinations which can be taken by examinees on computers hosted by the examination service on specified dates. To facilitate hosting, the examination service must define standard file formats for defining examinations and for defining the marking procedures.

The hosting service will also allow the assignment of unique identifiers for all examinations. This enables the following sequence of events:

Examination Supervision Service

The examination supervision service will maintain a fixed environment of rooms containing computers for examinees to take examinations on. The computers will be connected on a local network, and will also be indirectly connected to the Internet, to enable downloading of examinations (from creators) and uploading of final results.

Examination supervisors will identify examinees when they come in to take their examinations, and they will make sure that the examinees to do not cheat by using hidden materials or communication devices or by talking to each other.

We can say something about the details of the computer systems on which the examinations are taken:

Marking Services

Each examination will be defined as a two-part document. The first part consists of the examination as taken by the examinee. The second part consists of instructions for marking the answers. In most cases the marking instructions will consist of a simple list of possible answers for each question, or, in the case of questions to be subjectively marked, a flag implying that that is the case. More sophisticated options may allow for simple scripts to evaluate larger sets of possible answers to some questions.

It is useful to keep the two parts of the examination document separate for security reasons, so that there is less possibility of students having any access to marking criteria while they are doing the examination.

Results Distribution Services

The final stage in the examination process is to make the results available. The most straightforward way to do this is to post results on the web. To keep everything as transparent as possible, results should be posted together with the examination papers and the marking criteria, so that everyone can see exactly what level of difficulty the examination had, how well it covered the topic, and also whether the questions were well designed.

However, it is likely that in many cases examinees would prefer to have the option of only revealing their examination results to specified parties, and this could be achieved by requiring web accounts for anyone wishing to view results, and examinees would explicitly enable the viewing of results by selected accounts on the system. There will be also be other cases (discussed below), where the creator of the examination creates it for their own internal purposes, and does not wish it to be released to the general public.


Even where responsibilities are clearly defined, there will still be occasions where said responsibilities are not carried out, whether from carelessness, negligence or outright ill-intent. As a result, there will need to be options for people relying on examination results to question those results that turn out not to match the observed capabilities of the examinee, and, for example, where it can be determined that some particular examination centre produced many unreliable results in a certain period, it may be necessary to require examinees examined over that period to retake the examinations that they took.

Internal Examination Services

In some cases the creator of an examination will be the same organisation that is judging the results of the examination. So a company could use the services of a commercial examination service to administer and supervise a company aptitude test, which could be taken by potential employees without even requiring them to travel one particular location.


In many countries in the world today, education remains somewhat problematic, especially for the poorer classes. Although informal education is not for everyone, it offers flexibility and economy for at least some students. This potentially creates a demand for generic examination services, preferably with minimum possible cost, and an Internet-based computerised examination service and infrastructure is the most efficient way to satisfy this requirement.

Vote for or comment on this article on Reddit or Hacker News ...