Organising programming contests!!

During my college days, I organized Inscription, an international programming contest. Here are a few details on how to go about organizing a programming contest and the various things to look into.

1. Does your programming contest have a name? If not, get one - a catchy one!!

2. Have you decided on what scale you want to operate/organize? I mean, is it a international programming contest or regional or collegiate?

3. Decide on prize/reward to be given to winners. This depends on the answer to question 2 and your budget for the event.

4. Decide on the programming languages and corresponding compilers that need to be supported for the contest.

4. Decide on the contest date, timing and venue. Make sure all or atleast most of the contestants can participate.

5. Publicize the event on relevant Yahoo groups, Google groups, Orkut communities, etc. Send out mails to potential participants from earlier contest instances. Also, the website used for the contest could be submitted to search engines (http://art-support.com/URLsubmission.htm).

Do not send everyone a mail with all email IDs in the TO or CC list. Send them mails individually using some simple program or just BCC all of them. TO/CC are spammers dream. Don't help spammers get those email IDs so easily.

6. Keep the registrants informed about the happenings in the event. This includes telling them about practice runs of the contest, etc.

Further, there are 3 things one needs to take care of:

1. Rules/Contest format,
2. Problems and Solutions and
3. Infrastructure.

Rules/Contest format:

This is the first thing one needs to do! (meaning this is very very critical). Depending on the answer to question 2 above - decide on the rules of the contest. The most popular rule/contest format is that of ACM ICPC, but many other contests use their own custom formed rules. The ACM ICPC contest rules would require you to understand their rules properly. We feel that these rules are a bit hard to follow in-case you are short on time to prepare for the contest. For making your own rules - discuss with the rest of the group, be consistent with the rules, refer to rules provided by other tech-fest programming contests (Bitwise, Inscription etc). But from our experience, we feel that sticking to rules of ACM or other standard programming contests is the best.

Also, if you plan to have your contest on a particular platform, compiler, etc then specify it clearly (like GCC of a particular version or upto a particular version). Better is providing a good variety of platforms (Windows - Dev C++ / Linux - GCC). When doing this, a very important aspect is testing - check that all solutions that you have coded passes on all the platforms for all the languages supported.

Problems and Solutions:

1. This is the job of the problem setter. Problem statements form the heart of the contest. Start getting/setting questions and their solutions for the contest. Start early, since it is the most time consuming process. You have to get both the problem statement as well as the solution(s) (best one possible). If you are using a open-ended/research oriented question then do a lot of research on it and make sure the problem statement is solvable and there are no glitches. Doing a bit of research on the nature/background of the contestants who have attended your contest before helps here.

The problems are categorized as:

a) Level 0 (Easy) - problems here should be solvable by most of the contestants. These are aimed at novice programmers and those who are new to programming contest. These questions give them enough enthusiasm to go further to tackle harder problems. Without these type of problems most of them will just give up - which is not what you want. In general, first year B.Tech students have a lot of enthusiasm to take part in such contest. They are usually unfamiliar with anything but Turbo C/Borland C++. It would be a great plus if you can ensure that you can accommodate them as well (both in terms of questions and compiler compatibility)

b) Level 1 (Intermediate) - set a few tough ones. These problems are directed towards intermediate and novice programmers.

c) Level 2 (Hard) - the real tough ones go here. These problems are for those geeks and gods of programming!! But be careful here, if you do set such type of problems (which is not mandatory, but makes your contest interesting and popular) do a lot of research about it and ensure that the solution and problem statement are right. (Suggestion: involve your professors/lecturers/somebody)

2. For every problem in your contest decide on the time limit, memory limit and other constraints within which the solutions provided by the contestants has to execute (run your own solutions against this limit and make sure it passes through). Each question should have valid solution on all platforms and satisfy the evaluation criteria.

3. Make sure your test cases (test input and test output) are right. Don't mess them up.

4. Code all your solution (for all the problem statements) in all the programming languages that is supported by the contest. This comes in handy to make sure everything is working fine on the contest day.

5. Involve as many people as you can in choosing the problem statements and solutions (Suggestion: you should have a different team for this). Once you have set the questions, get it proof read by someone who was not involved in setting questions. Statements which are very trivial and obvious to the setter are not always so trivial to someone who reads it for the first time.

6. And don't copy problems from already existing contest sites (eg. Topcoder, etc). Its against their rules and involves copyright issues.

Infrastructure:

Generally involves 2 things mentioned below:

1. Access to the contest (the website in many cases - depends on answer to question 2)
2. Processing/Evaluating the solutions given by the contestants (Automated judging software or manual labor)

Website:

If your event is conducted over a network, then probably you would be creating a website to organize the event. This website could be used by the contestants to:

1. register during the registration period,
2. view rules/contest format,
3. view prize/rewards that would be given out to winners,
4. view important announcements, news and other information provided by the organizers,
5. view/download problem statements on the contest day,
6. submit solutions to various problem statements,
7. view ranking or status of all contestants,
8. interact with the organizers to clarify various issues that may crop up before and during the contest

A decent website should suffice, nothing hi-fi required. Look into websites of other programming contests to get ideas. Get the site ready as soon as possible - will be required for registration of contestants and helps in debugging the site itself. If possible have a forum sort of thing to interact with contestants before, during and after the contest. Get all the required information from the contestant, during registration and save this as it will be useful for the subsequent contests. Information - especially mailing address and email are required to communicate with the winners of the contest. Also, ensure that you do not post the contacts of the contestants on your website. Again this is a spammer's dream and the contestants will hate you for the rest of their lives.

Make sure you have good web server - this is a hardware issue. Think about the load on the machine during the contest (at Inscription '07 we ran into this problem and there was nothing much we could do about this). So prepare before hand - run some kind of stress test (eg. a trial run of the contest). Have backup plans here - a spare server to switch over to, or load balancing or something of such sort. Be ready for DOS attacks. There is always at least one evil guy with internet connection who doesn't want your contest to run successfully. That guy will start a DOS attack and you should have a filter for it or have some other contingency plan. Check your website for XSS (Cross Site Scripting - http://en.wikipedia.org/wiki/Cross_site_scripting) bugs. An outsider can use it to modify your database contents.

Online judge/Validator:

This is the best part! This is the brain of your contest!! The rules/contest format is usually coded into the judge. The judge receives solutions submitted by the contestants, filters it for malicious code, compiles it, executes it with the given test inputs, compares with the given test outputs and assigns points/credits to the contestants (http://en.wikipedia.org/wiki/Online_judge). It runs for the entire length of the contest, processing the solutions given by the contestants and updating the score board.

If you are following the ACM ICPC rules you can use - PC^2 (http://www.ecs.csus.edu/pc2/).

If you are not following the ACM ICPC rules and instead have your own custom made rules, then you would have to either:

1. write your own judge. Its not that difficult!,
2. you could use the "Judge" authored for Inscription. Its written in C++. But the catch here is that the rules have to be compliant with the ones used for Inscription or
3. you could modify the Inscription judge to suit your requirement (its GNU GPL2 licensed). Look at projects section of the website.

There are other online judges running on the Internet. They are used to practice for ACM ICPC or other such contests. If any of these could be useful in organizing a contest, please update this by commenting.

Tips and tricks:

1. make sure the whole system works a day before the event,
2. during the event, have a chat session opened with the contestants (infact have a email ID and messenger opened just for this purpose). This makes it much more interactive - fun for you and the contestants,
3. always keep in touch with the contestants - lots of bugs will be highlighted this way,
4. have couple of dry runs 2-7 days before the contest. A software always has bugs. The sooner you know them, the better it is.
5. have a few scripts/programs that come in handy. For example, many a times, the participants forget their password. Use a script to reset the password or even better have a web based password resetting thing in place,
6. use a test database (separate from the main-event database) for performing your tests before and during the contest day
7. do conduct practice runs - they are vital!
8. specify the compilers used by the judge, their version and the platform the judge is run on. Also, specifying the links where these compilers are available for download - makes it more comfortable for contestants
9. also, knowing the contestants is a big bonus as far as getting feedback is concerned and also basically because they are all good people - smart and intelligent; it always pays to know people in the intelligentsia.

Links:

Submitting website to search engines (http://art-support.com/URLsubmission.htm)
Online judge (http://en.wikipedia.org/wiki/Online_judge)
PC^2, follows ACM ICPC rules (http://www.ecs.csus.edu/pc2/)

Thats the most of it for now!!

PS - Thanks to Vishwajith, Mikhil, Abhradeep and Vivek for helping me finish this article.

top | post comment