2018 ICPC North America Qualifier
The NAQ is an online, pre-regional programming contest to help coaches choose teams for regional competitions (or serve as practice).
Post-contest Results
Congratulations to all the teams that participated!
- The official contest standings
- The open division contest standings
- The contest problems
- Slides of the Judge's solution explanations
- PDF of the problem set
- Judge's data (in .tar.xz format; 18 MB file)
Statistics on the official contest:
- Number of distinct Kattis users that submitted something: 832
- Number of universities actively participating: 117
- Number of teams that actively participating: 694
- Number of teams that solved at least one problem: 669
- Number of submissions during the 5 hours: 6839 (22.8 per minute)
-
Distribution of the number of problems each teams solved:
Number solved 1 2 3 4 5 6 7 8 9 10 11 Number of teams 49 278 188 59 33 24 20 7 5 4 2
Many thanks!
- Thanks to all contestants who participated!
- Thanks to all coaches who registered all those teams.
- Thanks to the judges who put in tons of work preparing the contest problems. In lexicographical order, the 2018 judges were: Anand Panangadan, Arup Guha, Bowen Yu, Etienne Vouga, Godmar Back, Greg Hamerly, Joshua Guerin, Kathleen Ericson, Mike Domaratzki, Peter Steele, Robert Hochberg, Yujie An, and Zachary Friggstad.
- Thanks to the reviewers who helped select the contest problems. In 2018, they were: Albert Chan, Amanda Holland-Minkley, Anand Panangadan, Bowen Yu, Bruce Elenbogen, David Sturgill, Elena Harris, Ethan McGee, Etienne Vouga, Godmar Back, Greg Hamerly, H. Keith Edwards, Howard Cheng, Howard Whitston, Ivor Page, Jeff Ifland, Jeffrey R. Paone, Joshua Guerin, Joshua Wang, Kathleen Ericson, Kathy Ames, Laura E. Brown, Liam Keliher, Manuel Alcántara Juárez, Marko Berezovsky, Martin Allen, Matthew Anderson, Mike Domaratzki, Nathan Backman, Nathan Gossett, Patrick Kinnicutt, Peter Steele, Robert Hochberg, Robert Trenary, Samuel Merten, Troy Vasiga, Uttam Thakore, Yujie An, and Zachary Friggstad.
- Thanks to the Kattis staff for supporting the contest.
Contest Week Information (updated 2018/10/05)
Here's some last-minute information. More recent information is at the top.
- The official NAQ contest is at https://naq18.kattis.com/ -- that is where officially registered contestants should participate on contest day.
- All Kattis accounts have been created and mailed out. You belong now to one of three groups:
- You did not have a Kattis account previously associated with the email address used for ICPC registration. In that case, you should have gotten an email with the new account. Use that for the contest. (even if you have other Kattis credentials).
- Same as above, but for some reason (e.g. spam filter) you did not get email with the new account. In this case, you need to get your password. Keep reading.
- You previously had a Kattis account with the email you used to register for the contest, and therefore did not get an email about any new account -- because you should use that existing account. However, you may have lost/forgotten your password. Keep reading.
- If you need immediate help before or during the contest (e.g. you cannot login), we have a Google Hangout for support for the 2018 ICPC NAQ. Please only use this if needed for Kattis login problems, after verifying that you are using the correct credentials and are at the correct site. This is not for problem discussion or problem clarification questions.
- IMPORTANT: Note that if a person has an existing Kattis account, but it is not affiliated with the email address used for ICPC registration, they would end up with *multiple* accounts. THEY SHOULD ONLY USE THEIR KATTIS ACCOUNT THAT IS AFFILIATED WITH THE ICPC REGISTRATION FOR THIS CONTEST. We have had many contestants who could not participate in past NAQ contests because they did not realize that they were logged in with an account that is not part of the contest. Also note, if a team has multiple contestants, any one may login to participate.
- There will additionally be a parallel (slightly delayed) Open Division contest with the same problems running at https://open.kattis.com/contests/naq18open This is a good place to send people who want to compete but are not registered (we will not add late entrants to the official contest). But people who participate in the Open Division will not show up on the official scoreboards, so only send non-official participants there. In particular, please do NOT send officially-registered contestants to the open division.
About the NAQ
The ICPC North America Qualifier is an online-only programming contest, scheduled to precede all ICPC North America regional contests. If you are a coach, you can use it in place of a local competition to help you determine teams for your regional ICPC contest. Or you can use it as extra practice. Please note: the North America Qualifier does not automatically qualify (or promote) any team to a regional competition.
The 2018 ICPC North America Qualifier will be on October 6, 2018, at 11:00 - 16:00 (CDT). It is an online contest, held on Kattis.
Register for the NAQ
Coaches, please register your teams on the Baylor ICPC registration site. At registration close, we will take this data and use it to create the teams on Kattis. Note that you must register for this contest to participate; registering for a regional does not register for this contest, or vice-versa.
Make sure you register your team for the contest titled "North America Qualifier". (There are other contests with the word "Qualifier" in the name, so be careful.) After registration is closed, due to the number of teams and the limited support, it will be impossible to accommodate new registrations or changes to existing registrations. So please register early.
There is one contest, with 12 "sites". The 12 sites correspond to the 11 regions of North America, plus one extra site for contestants who are not eligible to compete in ICPC in North America (e.g., high school students or people whose school is not in an ICPC North America region).
Please register for the site that corresponds to your region. Here are direct links for each region (using these is likely the simplest route to registration):
Coaches may register as many teams as they like (each team can have anywhere from 1 to 10 competitors — that's up to you). If you are in doubt about how many you may need, register more up front (as it will be impossible to add teams after registration freezes). But please try to be somewhat realistic for our planning purposes.
Coaches can create empty teams as placeholders but should fill them with the actual students that will compete before registration is closed. "Empty" teams will not get competition login credentials.
Frequently Asked Questions
- What is it? An online (distributed) programming contest, offered as a drop-in replacement for so-called "qualifying" contests (e.g., school-level, pre-regional competitions).
- When is the contest? — October 6, 2018, 11:00 - 16:00 (CDT).
- When does official registration begin? It's open.
- When does official registration close? Please see the official ICPC registration site.
- Can I register after registration is closed? No, due to the number of expected people participating and the limited support. Please make sure to register for the correct contest before the deadline.
- Where do I register? See the links above on this page.
- What contest should I (a coach) register my team for? The site corresponding to your region. Note that this is the "North America Qualifier", not a regional competition. As the names of different contests may look similar, please make sure you register for the North America Qualifier, and the site corresponding to your region.
- What if I am not a part of ICPC? Please register then for the ICPC-Ineligible site. This is for contestants who are (e.g.) in high school or are otherwise ineligible to participate in a North American ICPC region.
- Is this a qualifying contest, a practice contest, or something else? We are calling this a qualifying contest for the regional competitions. It can be (but is not required to be) used to help choose teams for the regional programming competition.
- Why is it being offered so early in the fall? To make sure it precedes all North American regional competitions.
- Must students participate in teams of three? Coaches may make teams of one to 10 contestants, but no prescribed number is required. The coach decides for their school and teams.
- Must I participate? No. If you don't find this useful for your situation, feel free to ignore it.
- I already offer a contest. Why should I use this one? There are several benefits. It will save you work, increase registration for your region (which helps in allocation of wildcard slots for the world finals), and gives all registered students a free year of ACM student membership.
- How will results be used? However, the coaches choose to use them. The results do not dictate anything about which contestants get to go to regionals, for example. However, one way we envision a coach may use the results is to select the top-finishing team(s) for the regional competition.
- Where should I go to compete? Please compete wherever you are. This is an online competition, so teams can meet in a common location, but coaches should organize that on their own.
- What resources does a local site need to provide? For each team, a computer with internet access, a web browser, and whatever development environment(s) you wish.
- What judging system will be used? Kattis. You can create a free account and try out Kattis at open.kattis.com. Note that the actual contest may be hosted at a different website with different login credentials (which your coach will receive via email after registration closes).
- Do I need to provide / can I provide my own judging? No, it will be done through Kattis and our centralized contest management.
- Can I provide my own problems? We welcome your offer to contribute problems to the official problem set — if you wish to do this, please contact Greg Hamerly. However, we will not support additional problems for just one site.
- What languages will be supported? C, C++, Java, and Python 2, Python 3, and Kotlin. Please note that Python 2 is accelerated via Pypy while Python 3 is not. Therefore Python 2 tends to run much faster.
- How many problems will there be? About 10-12.
- What will be the difficulty level of the problems? We are aiming for a wide range, with multiple easier problems for novice programmers.
- How will the contest run? Contestants will submit code and receive responses via the web. Coaches/facilitators should ensure the integrity of their own contestants (such as internet usage during the contest, appropriate books and materials, etc.).
- Is this one contest or multiple contests? While we will use one set of problems, offered at one time, each participating school will be in its own contest with its own set of results. Results will not be collated across schools.
- How much does this cost? It's free.
- Will there be a practice portion of this contest? You can practice using the Kattis system at open.kattis.com. You can create a free account and submit problems to the judging system, just like in the qualifier. The actual qualifier will be hosted at a different website with different credentials.
- Who should I contact with questions? Greg Hamerly
Write Problems for the NAQ
If you're writing problems for the NAQ, this information is for you. You can disregard this if you are a contestant.
The schedule for problem development is as follows (but is subject to change):
- Friday, May 25: email hamerly@cs.baylor.edu if you are willing to *review* problems for the contest.
- Monday, May 28: deadline to submit problem ideas here: NAQ problem solicitation. (These can be as short as a few sentences describing the ideas.)
- Monday, June 18: deadline for problem reviewers to rank and select a subset of about 15 of the previously submitted problem ideas for full development. Authors will be notified soon after.
- Wednesday, August 8: deadline for full problem development.
We are using the Kattis Problem format for developing problems. This is the format used for the ICPC world finals. It provides a structure for problem writeups, test data, solutions, and configuration. It automates problem verification (for solutions and anti-solutions), time limit determination, and finally installation. There are two parts: the problem format and the problem tools.
Kattis problem format
Here is some information on the Kattis problem format:- There is an introduction to developing problems in the Kattis problem format.
- Start by modifying this sample ("skeleton") problem package.
- The minimum things you need to provide for a problem are:
- Read and fill in the relevant problem.yaml fields — at least "author". Most are done for you already.
- Write the problem statement in problem_statement/problem.tex using simple LaTeX.
- Put sample and secret test data files in data/sample and data/secret using .in and .ans suffixes with corresponding prefixes (e.g. A.in corresponds to A.ans).
- Put at least one solution to the problem in submissions/accepted (preferably one in Java and one in C/C++)
- Other nice things to have in a problem are:
- submissions/wrong_answer — program(s) which give wrong answers on (some) included test data
- submissions/time_limit_exceeded — program(s) which take too long on (some) included test data
- submissions/run_time_error — program(s) which cause a run time error on (some) included test data
- data/*/*.desc files to describe each test (e.g. A.desc)
- data/*/*.hint files to give feedback to submitters that fail on that test case in non-contest environments (e.g. A.hint)
- data/*/*.{png|jpg} files to illustrate each test case to judges (e.g. A.png)
- Here is a very detailed guide to developing problems in the Kattis problem format.
- For more examples of problems formatted in the Kattis problem format, here is the problem set for the 2015 NCPC. Warning: this is a large download; about 22MB.
Kattis problem tools
After you're done writing your problem, you can use the Kattis problemtools software on a Linux machine to verify the problem (using "verifyproblem") or to compile a PDF or HTML version of your problem statement (using problem2pdf or problem2html, respectively).
The program "verifyproblem" does the following:
- timing and correctness of accepted submissions
- timing and correctness of wrong submissions
- validity of the input and output
- validity of the problem metadata
- checks that the problem description compiles
You can get the Kattis problemtools in two ways:
- Try first to install this precompiled Kattis problemtools package for Debian / Ubuntu on your Ubuntu or Debian Linux distribution. This was built on Ubuntu 18.04.
- If that doesn't work, you can download and build the Kattis problemtools package from the source on Github. Clone or download it, run "make builddeb", and then install the resulting package (which is placed in the parent directory).
- There is also a Docker image which is available here: hamerly/kattisproblemtools. Use tag "v1.20180426", which is what we are using in our automated verification process. That image contains an Ubuntu 18.04 image with Kattis problemtools installed.
Further problem-writing tips
- In this contest, we will likely have more than 1,000 contestants. With a contest this size, there's no time for correction once the contest starts. That means that the problems must be as bullet-proof as possible. Fortunately, the Kattis problem package helps this process immensely (see below). Also, please enlist the help of colleagues to proof-read, provide alternative solutions, create test data, etc.
- Each problem must be developed in the Kattis problem package format.
Here's a quick overview:
- problem_statement/problem.tex: statement in LaTeX (there's an example LaTeX document in the skeleton provided)
- data/sample/*.in, data/sample/*.ans: correspondingly-named sample data test file(s) (put them here, not in your problem description)
- data/secret/*.in, data/secret/*.ans: correspondingly-named secret data test files (e.g. random01.in, random01.ans; complete01.in, complete01.ans; etc.)
- input_format_validators/: at least one program (written in python, c++, java, etc.) that will validate the format of the input files. The program should exit with code 42 if all is correct; any other exit code is considered failure.
- submissions/accepted/: at least one program that solves the problem in the intended manner (preferably one in C++, one in Java)
- submissions/run_time_error/; submissions/time_limit_exceeded/; submissions/wrong_answer/: directories that can contain optional incorrect programs (for the reasons indicated by the enclosing directory names)
- All problem descriptions should follow the following conventions:
- Be simple and clear in your writing (remember: 1,000+ contestants). Many contestants will not be native English speakers or have the same backgrounds for cultural references.
- Programs should read only from standard input and write only to standard output. Don't use file-based I/O. (Don't explain this in the problem description, just assume contestants will do it.)
- Use sentinel-termination (looking for a special symbol indicating end of input) or case counting (giving the number of cases up front). Don't use EOF-terminated input, and try to avoid line-oriented input.
- Give explicit limits (lower and upper) on *every* input variable. Check those limits in the input validator.
- Give explicit types for everything where there might be a question (e.g. real-valued versus floating-point versus integer).
- Don't be overly explicit with output formatting, unless the goal of the problem is output formatting. Kattis is forgiving in how it judges correct output (it allows space differences, and can be configured to allow small relative or absolute floating-point differences). In particular, do not require the use of rounding.
- For creating test data and submissions:
- Make sure that test data covers the limits stated in the problem.
- Try to use one file per test case. This avoids the problem of needing to print "Case #:" at the beginning of each output line. If your problem needs to use many small test cases, then it's okay to put multiple cases in a file. Still, you can break up the cases into different types across multiple files.
- If you intend to require a certain type of solution (e.g. O(n*log*n) is fast enough but O(n^2) should be too slow), then create test data that will show the difference, and create programs in submissions/time_limit_exceeded/ that follow the too-slow approach. (Similar ideas apply for submissions/wrong_answer/ and submissions/run_time_error/.)
Contact
If you have questions, first please consult the FAQ on this page.
If after that you still have questions, please contact Greg Hamerly.