Ask people to list important tasks for airlines and airports — many answers won't be surprising. Safety and being on time will come up. Friendly service will be high on the list, and some lucky travellers might highlight cold and sparkling champagne in business or first class.

But for hardened operations specialists, the answers could be quite different. They will mention resilience and disaster recovery.

When airport systems collapse, things get gnarly quickly. Queues are endless. Bags can't move. And tempers flare.

Passengers are quick to tell their friends how bad it was. But they'll also mention anything impressive that airlines or airports did to recover. Robust disaster recovery processes are good for the brand, good for the business.

The earliest time in the passenger journey when things can go wrong is if check-in systems fall over. These tools, known in the trade as Departure Control Systems (DCS), are used by airlines to get flights away on time.

While DCS can be extremely complicated, they all have six features in common:

1. Receive passenger bookings for specific flights 
2. Allocate seats if they are not already purchased during booking
3. Issue a boarding pass and bag receipt
4. Print tags for the bags
5. Board passengers and bags at the gate, ensuring the correct people board with the complete list of names and offloading any missing passengers
6. Confirm the passengers that have been flown for the airline’s revenue accounts.

Airlines and their IT suppliers have added many layers of complexity to DCS' six core tasks. Frequent flyer status needs to be verified for priority check-in and security, as well as for granting access to fancy lounges. Special upgrade offers may be available. And seats with unique features may be reserved for families or high-fare payers.

Real-world implications of DCS failures

As operated, DCS can be so complex that when things fall over, it isn't just a simple matter of turning it on again.

All the extra functionality can stay down for long periods and be hard to recover. This happened to London-based British Airways in May 2017. Over 75,000 bank holiday travellers were stranded, and the airline cancelled 726 flights because someone unplugged a computer. This is by no means an isolated case. There are many such cases with other airlines.

Meanwhile, system vendors refer operators to their service level agreements. These specify the time that a system must be available. 99.95% (4.4 hours of downtime a year) is a typical figure. As long as the core system is down for less than 4.4 hours, vendors will claim they've done their bit. Even if the system is unusable for some reason.

System failures can occur due to various reasons, such as power outages, network issues, software bugs, hardware failures, or even problems with the common equipment used by airports to serve multiple airlines. These issues can cause a DCS to collapse, just like any other system.

How to recover from operational issues. Trials at JFK Terminal One

A recent trial that Ink conducted with airlines serving Terminal One at New York’s JFK airport is a good example of how to recover from issues. Air France, Air New Zealand, and Korean Air wanted to achieve four things:

1. Check-in and board passengers without access to their primary DCS
2. Re-start their hardware devices and apps within 15 minutes of a failure
3. Access the list of all passenger names applicable to their flights without DCS
4. Understand training requirements to perform check-in, issue bag tags, and board a plane within 30 minutes without their main DCS.

These four things are not as simple as they appear at first glance. Seat maps need to be defined and created in the substitute DCS (known as the Disaster Recovery System, or DRS) provided by Ink. Visas must be checked and advance passenger information must be transmitted to arrival airports.

The list of passenger names on a flight isn't necessarily the same as originally listed in an airline’s Computer Reservation System (CRS) or Passenger Service System (PSS). Some passengers may have misconnected or bought flexible tickets. A DRS needs to be able to handle re-accommodated passengers because the situation that brought it into use may also have affected people's bookings. 

Sometimes, the list of passengers booked on a flight may be unavailable or transmissible. This occurred on a flight during Ink’s JFK Terminal One exercise. A Passenger Name List (PNL)  was manually created using a process called NOREC (i.e. No Record).

No Record. No problem. Only 2.3% of passengers were able to be automatically checked in through the automatic Passenger Name List (PNL). As for the remaining 97.7%, airport staff used Ink's hand-held DRS devices to create a manual PNL. To identify these passengers, airline staff conducted a thorough search including their names, PNR, boarding pass‌ and passport scans. Using document scan functionality, which captures an image of their passport, agents collected their details such as name, surname‌, and APIS fields. The process was completed by manually adding cabin class and seat. Any seat change automatically caused a new boarding pass to be printed.

It took between 28 and 30 seconds to check in passengers, and 50 passengers per hour were processed on average. Transaction times can be much lower, depending on the airline's standard operating procedures. Boarding times ranged from three to six seconds per passenger.

Getting passengers through the airport isn't the only task for Ink DRS. It must also connect to the airport’s Baggage Reconciliation System (BRS), which ensures that bags are not loaded onto a plane without an accompanying passenger — an essential security measure.

The airline trade association IATA has defined standard baggage message formats, including Baggage Service Messages (BSM), so the two systems can communicate. Ink DRS creates bag tags for flights in accordance with standard flight number, date, time‌, and passenger details. As bags moved through JFK Terminal One, the airline could see each piece processed as BSMs were sent and received.

Ink DRS has the ability to print both inactive and active bag tags for regular passengers and special crew, rush, and gate tags when required. Before a tag is printed, it's labelled as ‘unprinted’ on the hand-held device.

Ink’s Chief Delivery Officer, Ivan Jakovljevic, explains that there are five core actions that airlines can take for their DRS to take control quickly and effectively should things go wrong:

1. Ensure that airport agents have access to the Passenger Name List (PNL) from CRS/PSS to keep check-in flowing smoothly.
2. Enable Interactive Advance Passenger Information within DRS whenever possible. Ink’s DRS can be certified with any government body to maintain compliance and avoid fines in a disaster scenario.
3. Provide complete documentation regarding applicable codeshare and interline agreements for airport staff. This will enable them to correctly re-accommodate passengers who are not on the PNL but are correctly ticketed to fly.
4. Provide documentation regarding their cabin Layout of Passenger Accommodation (LOPA) to each airport so teams can recreate accurate seat maps quickly and accurately in Ink DRS.
5. Encourage their airport representatives to be open and honest with passengers about why delays or issues are occurring and the impact this may have on their journey.

Four strategies for disaster recovery

Airline and airport expert Javed Malik adds four operational best practices to help airports manage disruptions effectively:

1. Equipment should be stored, charged, configured, and tested periodically, with malfunctions fixed or replaced and all relevant processes documented in manuals.
2. Airports should always have disaster recovery ‘champions’ on their team, available to ‘train the trainer’ and support disruptions if and when they occur.
3. Live tests should be completed regularly so that everything is always as ready for an outage as it can be. Some airlines may prefer to live test with DRS directly, but others choose to shadow their main DCS.
4. When using DRS, teams should try to get passengers as close as possible to their final destination. This can be achieved by adding interline connections. The priority is to get passengers and their bags to the next station, where they can be re-checked. However, passengers find it inconvenient to have to re-check downroute.

Putting things right when they've gone wrong can be the best investment for airlines and airports. Almost all airlines will experience check-in system issues from time to time. Fortunately, airlines can prepare with a DRS and robust supporting processes in place. Passengers will thank those who do.