Stage 1
Components Paper Submission
The Components Paper entry will be created and submitted via the contestant portal.
Section
Cover Page and Abstract
Word/Page Limit
Form Fields & 1,000 words
Description
Form fields include the following:
- Contestant name (individual, team, organization, company)
- Application title
- Technical and business points of contact (name, phone, address, email address)
Contestant abstracts shall describe succinctly (1,000-word MAXIMUM):
- The proposed technology contribution.
- How it can improve performance in the stated application.
Research Paper
10 pages (IEEE 2-column format)
Contestants shall describe their proposed technology contribution and how it can contribute to the stated Operational Use Case applications. Competition organizers will provide an example Call for Papers at the outset of the stage.
Summary Slide
1 page
Contestants shall provide a single slide (text no smaller than 10 point) in PDF format summarizing the proposed project. A quad chart layout is recommended.
NOTE: Submission(s) must not use NIST’s logo or official seal and must not claim NIST endorsement.
Stage 2
Test Method(s) Measurements and Attestation Video Submissions
For a series of operational and technical tests, contestants will build their own copy of the required, low-cost and easy-to-build test method apparatuses deployed in their own space (e.g., at a campus, stadium, flying club), which will include several ground sensors and a ground control station. Contestants will fly and record videos of their UAS completing tests according to instructions provided by NIST directly following the Stage 2 webinar.
All registered contestants will record test method(s) measurements and attestation video themselves, and provide their test method(s) measurements and attestation video to NIST via the contestant portal for review and verification prior to leaderboard upload and prior to the contest closing date. Detailed instructions on video content will be provided at Stage 2 launch.
This section lists the various Stage 2 specifications. See the Glossary for specific definitions of terms used. Competition organizers will also provide additional resources at the outset of the stage to assist in the interpretation of these specifications.
| Requirement Title | Requirement Definition | Required Materials |
| Part 107 Compliant | Contestants shall comply with the requirements for operating their UAS under FAA Part 107 Regulations, in Class G airspace, during the day, over an unpopulated area, without needing any additional authorizations or exemptions. | Team Statement and Supporting Documentation |
| Battery Powered | UAS, including the Uncrewed Aerial Vehicle (UAV), Base Station, Supervisory Controller, and associated equipment, shall be powered only by batteries. | Team Statement and Bill of Materials |
| Supervisory Controller | Provide a Supervisory Controller that, in the hands of a Part 107 certified person and in a location where they can observe the entire flight area, satisfies the requirements of the Remote Pilot in Command according to FAA Part 107, up to a distance of 300 feet, with no additional exemptions or authorizations. | Team Statement and Supporting Documentation |
| Positive Aircraft Control using the Supervisory Controller | Demonstrate positive aircraft control of the UAV using the Supervisory Controller by performing the Inspect (MAN 4) test of the Open Test Lane:
|
Test Method(s) Results and Attestation Video |
| Lost Communications Behavior |
If the UAV loses communication with the Supervisory Controller, it shall stop its mission and land safely. Before doing so, it may optionally:
|
Team Statement and Documentation |
| Low Battery Behavior |
If the UAV enters a low battery state, apart from during the Endurance test, it shall stop its mission and land safely. Before doing so, it may optionally:
The UAV shall enter a low battery state while it still has enough battery power to perform all of the following, at an ambient air temperature of at least 50ºF (10ºC):
This will be tested as part of the Endurance test. |
Test Method(s) Results and Attestation Video |
| Bill of Material (BOM) Total Cost | The total build price of the complete, ready-to-operate system, including all components (vehicles, control stations, computers, software, etc.), and any fees (e.g., subscription, license) necessary for operation shall be less than $20,000. If applicable, all fees must include at least six months of access to the subscribed or licensed services in the total cost. | BOM Spreadsheet |
| Safety Plan specific to the team’s UAS |
In acknowledgement of the wide variety of UAS types, and associated risks, contestants will submit their Safety Plan. At a minimum, the Safety Plan shall include:
To meet the requirements in Stage 2, the Safety Plan will only be evaluated based on the above headings with approval by competition organizers. |
Team Statement and Documentation |
Metrics: Measurement of Capabilities – Whole System (also used for Optional Mid-Stage Progress Review)
Contestants will be evaluated on the basis of the following metrics. Competition organizers will provide additional resources on Open Test Lane and Test Methods at the outset of the stage.
| Measurement | Description | Metric to Maximize | |
| 1 | Inspect and Download Data |
Perform the 10 minute Inspect Payload test of the Open Test Lane (PAY) while downloading data from the Target Objects of Interest. Transmit collected imagery and data to the Command Server. The Open Test Lane will be set up as follows:
|
Maximize the amount of unique data downloaded from the Target Objects of Interest and transferred to the Command Server. There is no advantage to achieving a level of acuity beyond that specified. For every minute (rounded up) that human intervention is required, 10% will be deducted from the test score, up to a maximum of 50%. |
| 2 | Survey Acuity |
Perform the 10 minute Embedded Survey Acuity Test while downloading data from the Target Objects of Interest. Transmit collected imagery and data to the Command Server. All Target Objects of Interest must be inspected to a level of detail that will be specified in the procedure, approximately equivalent to the ability to resolve 3.5 mm (0.138 in) features. |
Maximize the amount of data downloaded from the Target Objects of Interest and transferred to the Command Server. There is no advantage to achieving a level of acuity beyond that specified. For every minute (rounded up) that human intervention is required, 10% will be deducted from the test score, up to a maximum of 50%. |
| 3 | Autonomous Obstacle Avoidance | Autonomously fly between two points, avoiding an obstacle in between them. | Maximize the number of iterations in 10 minutes without hitting the obstacle. |
| 4 | Endurance (time) | Perform the Test Method for Endurance, aiming to fly for as long a duration as possible while completing at least one iteration. |
Maximize flight time on one battery charge. There is no advantage to completing more than one iteration. |
| 5 | Endurance (distance) | Perform the Test Method for Endurance, aiming to fly for as many iterations as possible. |
Maximize the number of iterations of the test on one battery charge. The flight time is ignored. |
Metrics: Best-in-Class
The best performing contestant in each of the following subsets of the Main Contest tests (some maximized with respect to different metrics) will receive respective Best-in-Class awards. They will also receive an invitation to demonstrate their capabilities in Stage 3 if they are not otherwise selected on the basis of the leaderboard ranking.
The purpose of the Best-in-Class awards is to encourage the participation of contestants who may have a specialization that meets a specific need, even if they do not have the resources to field a system that can compete in all the competition tests. Therefore, NIST PSCR encourages contestants who are only focused on one of the following subsets of tests (in addition to the Required Specifications) to participate.
|
Capability |
Description |
|
|
1 |
Best-in-Class Endurance |
Achieve the most iterations in the Endurance (Distance) test over one battery charge. |
| 2 |
Best-in-Class Inspection and Survey Acuity |
Inspect, to the highest level of detail, the largest number of targets across all Target Objects of Interest, across the Inspect and Download Data, and the Survey Acuity tests. Amount of data downloaded and human intervention penalties are ignored. |
| 3 |
Best-in-Class Radio |
Download the most amount of data across all Target Objects of Interest across the Inspect and Download Data, and the Survey Acuity tests. Inspections and human intervention penalties are ignored. |
Capability 1 of the Best-in-Class Contest is identical to Measurement 2 of the Main Contest. Each team’s latest result for Measurement 2 of the Main Contest is therefore automatically considered for Best-in-Class Capability 1.
While Best-in-Class Capabilities 2 and 3 make use of some of the same Measurements as the Main Contest, contestants may score better by flying in a different manner to the way that they did in the Main Contest. For example, a team may maximize their Best-in-Class Capability 2 score by spending equal time inspecting each Target Object of Interest to their best level of acuity. In contrast, in the Main Contest they might aim to only achieve the required level of acuity and instead focus their time on downloading from Target Objects of Interest that have radio transmitters. Therefore, contestants who wish to be considered for Best-in-Class Capabilities 2 and 3 shall submit separate results and accompanying videos for each of the two Best-in-Class capabilities’ tests in each of Best-in-Class contests 2 and 3. These may also be resubmitted as many times as contestants wish, with the latest results being considered.
Note that contestants must use the same UAV, in the same configuration, for their Main Contest and Best-in-Class Contest entries. A team who updates their UAV to a new configuration for one contest must re-submit new measurements.
Stage 3
This section lists the various specifications for Stage 3. See the Glossary for specific definitions of terms used. Competition organizers will also provide additional resources at the outset of the stage to assist in the interpretation of these specifications.
Contestants must comply with the same Required Specifications as listed in Stage 2 with the following adjustments.
Measured Specifications and Capabilities
The Stage 3 Contest will see contestants rotate through a test method station and one or more scenario stations. Each station results in a team score, which is normalized to the highest score at that station. The leaderboard is based on the sum of each team’s normalized scores across the stations.
At the test method station, contestants will perform, live, the tests they performed for the Stage 2 Metrics for Main Contest, and their score will be computed accordingly. They may perform as many of the tests, as many times as they wish, within the allotted time. The best measurement for each test will count.
At each scenario station, contestants will fly a mission in a scenario that resembles a public safety use case, relevant to the challenge application. For example, the scenario might be a simulated mass casualty event in an urban area, with simulated victims (mannequins) in and among a collapsed structure, cars, and so-on. There will be a number of Target Objects of Interest in the environment, in locations that are operationally relevant, such as next to the simulated victims. Some may be out in the open, some may be in obstructed or confined areas. The radio transmitters at the Target Objects of Interest will include similar capabilities to the ones contestants build for their own testing in Stage 2. They may have additional capabilities for measuring connection quality. Details and sample code for contestants to perform their own testing will be made available prior to the launch of Stage 3.
The scenario may require the UAV to fly beyond line-of-sight and/or radio communications of the Base Station. (The Remote Pilot in Command, with the Supervisory Controller, will be in a separate location where they can maintain communication and visual line-of-sight to the UAV.) Over the course of the allotted time, the team shall maximize the number of target items read/identified, and maximize the amount of data gathered from the Target Objects of Interest and transferred to the Command Server. The scoring formula will incentivize the collection of data from multiple Target Objects of Interest, rather than just focusing on collecting data from one.
Contestants will be allowed five minutes of hands-on time at the start of the mission to set up and launch their system from a stowed state. For every additional minute (rounded up) that human intervention is required during the rest of the mission (meaning any input from any human to the system, via any modality), 10% will be deducted from the measurement for that test before computation of the normalized score, up to a maximum of 50%. Administrative intervention, such as addressing another aircraft that has unexpectedly appeared in the airspace, is exempt from this penalty.
Contestants may visit stations more than once as scheduling permits. Only their highest score in each station is counted. Each run of a given test station be as if it was approached anew, contestants shall not use information gathered prior to the current mission (e.g. during practice, or a previous mission in the same station), such as maps and waypoints. Competition organizers may take measures, such as adjusting the station periodically while keeping the difficulty level the same, to prevent and detect re-use of previous information.