CRM Software for Economic Development Organizations
+1 (805) 967-1280 | email: sales@outreachsystems.com

Understanding the APEX Accelerator Goaled Metrics 1.2 Scorecard

The APEX Accelerator Performance Metric Scorecard is based upon the DoD GOALS AND METRICS 1.2 document released February 2023. The scorecard is broken out by each of the goals set forth by DoD. The scorecard draws from counseling session, milestone session, award session and training session records along with entries in the APEX Accelerator Metrics Log:

With respect to those line items that are based upon activity records, only reportable activity will be included on the scorecard, and the date of the activity must fall in the reporting period of the scorecard and must be assigned to one of the funding source(s) and center(s) selected on the scorecard definition.

This scorecard also includes a number of line items that do not take activity into consideration. For example, the number of DIB-ready clients simply counts those clients that have a DIB-ready date that falls between the scorecard start date and the scorecard end date. The client's counseling session may be scheduled to take place the following week but that date could be in a different quarter. Because these line items merely look at the client record, the funding source for these line items is not taken into account.

This scorecard cannot be run on a per counselor basis. This scorecard is designed to be run against your entire program. Furthermore, this scorecard includes various line items from the APEX Accelerator Metrics Log. Line items from this log cannot be run with audit information turned on. In other words, the Processing Mode cannot be set to "Include audit information" when this log is referenced.

And, of course, it is important to note that this scorecard does not take the "DLA Active" parameter into consideration. There is no minimum amount of time you must spend with a client in order for that client's activity to be included on the scorecard. Furthermore, all reportable training events are considered for reporting purposes, even those that do not have the "Participated by Center" checkbox turned on.

Two condensed versions of this scorecard are also available called the APEX Accelerator Goaled Metrics scorecards. One is based upon the 1.1 goals, while the other one is based on the 1.2 goals. These condensed versions only include those line items that are goaled per DoD. The logic associated with the line items on the APEX Accelerator Goaled Metrics scorecard is identical to the logic of the corresponding line items described below.

Goal 1: Cultivate the Defense Industrial Base (DIB) and Government Industrial Base (GIB)

This first section provides a tally of all clients that are newly DIB or GIB-ready, or both. Since there is no clear way for Neoserra to measure clients that are submitting bids and thus attempting to re-enter the DIB or GIB, this information can be logged in the APEX Accelerator Metrics Log and thus be included in the scorecard.

It should also be noted that these two line items count all clients with a DIB/GIB-ready date that falls within the reporting period. These clients may, or may not, have been counseled or trained yet. Because DoD only wants to count newly DIB/GIB ready clients once, some clients may be counted before they meet with a counselor. For example, you may have a brand new client that is DIB-ready when they walk through the door on March 27th. However, you may not have a meeting scheduled with this client until April 15th. If you wait to report this client until their counseling session, then they won't be considered "new" anymore because their readiness date fell in the prior quarter. As such, the Neoserra reports are based upon the readiness date, not the counseling session date.

Number of new DIB-ready clients This line item provides a tally of unique clients whose "Defense Industrial Base (DIB) ready?" checkbox has been set to Yes, and whose "DIB Ready Date" falls between the Scorecard Start Period and the Scorecard End Period. This line item will count clients across ALL centers selected in the scorecard definition record, but this line item does NOT take funding source into consideration. As such, clients assigned to non-APEX centers, who have had no association with the APEX program, may be included in this line item if the DIB-ready date for the client falls within the reporting period and the client's center was selected on the definition record.
Number of new GIB-ready clients This line item provides a tally of unique clients whose "Government Industrial Base (DIB) ready?" checkbox has been set to Yes, and whose "GIB Ready Date" falls between the Scorecard Start Period and the Scorecard End Period. This line item will count clients across ALL centers selected in the scorecard definition record, but this line item does NOT take funding source into consideration. As such, clients assigned to non-APEX centers, who have had no association with the APEX program, may be included in this line item if the GIB-ready date for the client falls within the reporting period and the client's center was selected on the definition record.
Number of clients attempting to re-enter the DIB and GIB This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of clients attempting to re-enter the DIB and GIB (e.g., the number that submitted bids)."
Number of DIB and GIB-ready clients assisted (not included above) This line item will calculate all clients that have been counseled during the reporting period who are either GIB or DIB-ready but whose DIB or GIB-readiness date does not fall within the scorecard reporting period. In other words, these are all "old" DIB and GIB-ready clients that you have worked with during the reporting period.

And then here is a visual representation of where the data is coming from:

Goal 2: Increase Equity and Inclusion

The second goal section in the scorecard provides a breakdown of clients based upon their socioeconomic certifications. The DoD GOALS AND METRICS document specifically calls out:

  • SDB Clients
  • HUBZone-certified Clients
  • SDVOSB Clients
  • WOSB Clients

The Neoserra scorecard additionally provides information for the following socioeconomic certifications that are not specifically included in the DoD GOALS AND METRICS document:

  • VOSB Clients
  • VOSB and SDVOSB Clients combined
  • EDWOSB Clients
  • WOSB and EDWOSB Clients combined

There are three line items for each of these socioeconomic certifications. It should be noted, that these line items look for clients who are newly certified, not newly DIB/GIB-ready. In other words, only clients that received their HUBZone certification within the current reporting period will be counted in the respective HUBZone line items.

Furthermore, it should be noted that the "Number of new..." line item for each of the socioeconomic categories counts clients with a certification date that falls within the reporting period. These clients may, or may not, have been counseled or trained yet. Because DoD only wants to count newly certified clients some clients may be counted before they meet with a counselor so as to ensure that they get reported, as such this line item does NOT take funding source into consideration. Clients across ALL centers selected in the scorecard definition record, will be included.

In red are those underserved communities not specifically called out in the DoD document:

Underserved Community Number of new ... Number of contract awards to ... Amount of contract awards to ...
SDB Any client with a Disadvantage Certification Date between the scorecard start and end date will be included in this line item calculation. This line item will provide the total number of awards with a reporting date that falls within the scorecard period, won by clients with one of the following Disadvantaged Statuses: DBE Certified, MBE Certified, SBA 8(a) Certified or Self-certified Disadvantaged/Minority Owned. This line item will provide the total dollar amount of awards with a reporting date that falls within the scorecard period, won by clients with one of the following Disadvantaged Statuses: DBE Certified, MBE Certified, SBA 8(a) Certified or Self-certified Disadvantaged/Minority Owned.
HUBZone Any client with a HUBZone Certification Date between the scorecard start and end date will be included in this line item calculation. This line item will provide the total number of awards with a reporting date that falls within the scorecard period, won by clients that are HUBZone Certified. This line item will provide the total dollar amount of awards with a reporting date that falls within the scorecard period, won by clients that are HUBZone Certified.
SDVOSB Any client with a "Company Veteran Status" set to "SDVOSB Certified" and a "Veteran Verification/Certification Date" between the scorecard start and end date will be included in this line item calculation. This line item will provide the total number of awards with a reporting date that falls within the scorecard period, won by clients with a Company Veteran Status set to either SDVOSB Certified or Service-Disabled Veteran. This line item will provide the total dollar amount of awards with a reporting date that falls within the scorecard period, won by ready clients with a Company Veteran Status set to either SDVOSB Certified or Service-Disabled Veteran.
VOSB Any client with a "Company Veteran Status" set to "VOSB Certified" and a "Veteran Verification/Certification Date" between the scorecard start and end date will be included in this line item calculation. This line item will provide the total number of awards with a reporting date that falls within the scorecard period, won by clients with a Company Veteran Status set to either VOSB Certified or Veteran. This line item will provide the total dollar amount of awards with a reporting date that falls within the scorecard period, won by clients with a Company Veteran Status set to either VOSB Certified or Veteran.
VOSB/SDVOSB Any client with with a "Company Veteran Status" set to either "VOSB Certified" or "SDVOSB Certified" and a "Veteran Verification/Certification Date" between the scorecard start and end date will be included in this line item calculation. This line item will provide the total number of awards with a reporting date that falls within the scorecard period, won by clients with a Company Veteran Status set to either SDVOSB Certified, VOSB Certified, Service-Disabled Veteran or Veteran. This line item will provide the total dollar amount of awards with a reporting date that falls within the scorecard period, won by clients with a Company Veteran Status set to either SDVOSB Certified, VOSB Certified, Service-Disabled Veteran or Veteran.
WOSB Any client where the "Ownership Gender" field is set to "Woman-Owned (WOSB) Certified" and where the "Ownership Gender Certification Date" is between the scorecard start and end date will be included in this line item calculation. This line item will provide the total number of awards with a reporting date that falls within the scorecard period, won by clients with the Ownership Gender field set to Woman-Owned (WOSB) Certified. This line item will provide the total dollar amount of awards with a reporting date that falls within the scorecard period, won by clients with the Ownership Gender field set to Woman-Owned (WOSB) Certified.
EDWOSB Any client where the "Ownership Gender" field is set to "EDWOSB Certified" and where the "Ownership Gender Certification Date" is between the scorecard start and end date will be included in this line item calculation. This line item will provide the total number of awards with a reporting date that falls within the scorecard period, won by clients with the Ownership Gender field set EDWOSB Certified. This line item will provide the total dollar amount of awards with a reporting date that falls within the scorecard period, won by clients with the Ownership Gender field set to EDWOSB Certified.
WOSB/EDWOSB Any client where the "Ownership Gender" field is set to either "Woman-Owned (WOSB) Certified" or "EDWOSB Certified" and where the "Ownership Gender Certification Date" is between the scorecard start and end date will be included in this line item calculation. This line item will provide the total number of awards with a reporting date that falls within the scorecard period, won by clients with the Ownership Gender field set to either WOSB Certified or to EDWOSB Certified. This line item will provide the total dollar amount of awards with a reporting date that falls within the scorecard period, won by clients with the Ownership Gender field set to either WOSB Certified or to EDWOSB Certified.

In addition to the socioeconomic certifications, this section also tracks the following metrics:

Number of trainings for Small Business Liaison Offices (SBLOs) This line item draws from the training event module in Neoserra and looks for all non-canceled events that have "Small Business Liaison Offices (SBLO) Training" as a primary training topic.
Number of matchmaker events This line item draws from the training event module in Neoserra and looks for all non-canceled events that have a Training Format of "Matchmaker".
Number of direct referrals and businesses resulted in success This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of direct referrals and businesses resulted in success."
Number of client training events on the DoD MPP This line item draws from the training event module in Neoserra and looks for all non-canceled events that have "DoD Mentor-Protégé Program Information" as a primary training topic.
Number of client training events on the Small Business Administration (SBA) MPP This line item draws from the training event module in Neoserra and looks for all non-canceled events that have "SBA Mentor-Protégé Program Information" as a primary training topic.
Number of clients with a new DoD MPP agreement This line item draws from the milestone module in Neoserra and counts all clients that have a "DoD Mentor-Protégé Program (MPP) Agreement" milestone dated during the scorecard reporting reporting period.
Number of clients with a new SBA MPP agreement This line item draws from the milestone module in Neoserra and counts all clients that have a "SBA Mentor-Protégé Program (MPP) Agreement" milestone dated during the scorecard reporting reporting period.

While the DoD Document also calls out the following metric: "Number of direct referrals and businesses resulted in success" this information is not tracked in Neoserra but can be recorded on the Metrics Log and thus included on the scorecard.

The following graphic explains the line items visually:

Goal 3: Increase Awareness of and Compliance with Foreign Ownership, Control, or Influence (FOCI)

There are five line items in this section of the Neoserra scorecard, however, two of the line items are not specifically called out for in the DoD Document and are merely offered here as supplemental FOCI information. The extra line items are show in red below:

Number of hosted and/or supported FOCI webinars and/or training events This line item draws from the training event module in Neoserra and looks for all non-canceled events that have "Foreign Ownership Compliance (FOCI)" as a primary training topic.
Number of FOCI publications provided such as newsletters, articles, and any press releases This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of FOCI publications provided such as newsletters, articles, and any press releases"
Number of clicks on online FOCI publications This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of clicks on online FOCI publications"
Number of FOCI counseling sessions This line item draws from the counseling session module in Neoserra and counts all counseling sessions where the Counseling Area has been set to Foreign Ownership, Control or Influence (FOCI) Compliance.
Number of clients counseled on FOCI This line item draws from the counseling session module in Neoserra and reports all unique clients that have received counseling where the Counseling Area has been set to Foreign Ownership, Control or Influence (FOCI) Compliance.

And for those who prefer a visual guide to the source of the data:

Goal 4: Improve Cybersecurity of the DIB and GIB

The line items in this section relate directly to the DoD GOALS AND METRICS 1.1 document, however, two of the metrics in the document are not natively tracked in Neoserra (Number of cybersecurity publications provided such as newsletters, articles, and any press releases; and Number of clicks on online cybersecurity publications) but they can be entered in the APEX Accelerator Metrics Log and then pulled into the scorecard.

Number of clients that have become compliant with CMMC 2.0 Level 1 This line item draws from the Neoserra milestone module and counts all clients that have a "CMMC Level 1 Certification" milestone dated during the scorecard reporting period.
Number of clients that have become compliant with CMMC 2.0 Level 2 This line item draws from the Neoserra milestone module and counts all clients that have a "CMMC Level 2 Certification" milestone dated during the scorecard reporting period.
Number of clients that have become compliant with Federal Acquisition Regulation (FAR) 52.204-21 This line item draws from the Neoserra milestone module and counts all clients that have a "Federal Acquisition Regulation (FAR) 52.204.21 Compliance" milestone dated during the scorecard reporting period.
Number of hosted/cohosted Project Spectrum (PS) training events This line item draws from the Neoserra Training Event module and counts all non-canceled events that have a primary training topic of "Project Spectrum Training".
Number of cybersecurity webinars and training events This line item draws from the Neoserra Training Event module and counts all non-canceled events that have a primary training topic of "Cybersecurity Assistance".
Number of cybersecurity publications provided such as newsletters, articles, and any press releases This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of cybersecurity publications provided such as newsletters, articles, and any press releases."
Number of clicks on online cybersecurity publications This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of clicks on online cybersecurity publications."

And, the graphical representation of the data sources:

Goal 5: Facilitate Innovation for the DIB and GIB

This section focuses on Government-led Innovation Programs (GIPs) and includes the line items referenced in the DoD Document.

Number of prep+contact hours on GIPs This line item adds all prep and contact time of those counseling sessions where the Counseling Area is set to: "SBIR/STTR/Other Innovation Programs."
Number of counseling sessions on GIPs This line item counts the total number of counseling sessions where the Counseling Area is set to: "SBIR/STTR/Other Innovation Programs."
Number of clients counseled on GIPs This line item counts the unique clients who have received counseling where the Counseling Area is set to: "SBIR/STTR/Other Innovation Programs."
Number of referrals that connect clients to federal laboratories This line item counts the number of times clients have been referred to a Federal Innovation Laboratory subsequent to a counseling session. This line item only draws from the counseling session referral field, not the client field.
Number of proposals submitted to GIPs by clients who received technical assistance This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of proposals submitted to GIPs by clients who received technical assistance."
Number of training events conducted on GIPs This line item counts the number of non-canceled training events where the Primary Training Topic is set to "SBIR/STTR/Other Innovation Programs".
Number of proposals won in GIPs This line item counts the total number of awards won by clients where the Contract Type is set to either SBIR Award or STTR Award.
Dollar amount won in GIPs This line item counts the total dollar amount of awards won by clients where the Contract Type is set to either SBIR Award or STTR Award.

And the corresponding visual representation of the various data sources for these calculations:

Goal 6: Strengthen the Supply Chain

The four line items in this section correspond to the DoD template for Goal 6.

Number of training events related to the DPA Title III This line item draws from the training event module in Neoserra and it counts all non-canceled training events where the Primary Training Topic is set to "Defense Production Act (DPA) Title III Support".
Number of clients submitting white papers to Funding Opportunity Announcements (FOAs) as a direct result of receiving technical assistance This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of clients submitting white papers to Funding Opportunity Announcements (FOAs) as a direct result of receiving technical assistance."
Number of clients submitting full proposals to FOAs as a direct result of receiving technical assistance This line item provides the sum of entries from the APEX Accelerator Metrics Log where the APEX Metric is set to "Number of clients submitting full proposals to FOAs as a direct result of receiving technical assistance."
Number of training and other events related to the IBAS This line item draws from the training event module in Neoserra and it counts all non-canceled training events where the Primary Training Topic is set to " Industrial Base Analysis and Sustainment (IBAS)".
Number of referrals to Workforce Dev. and/or Career and Technical Education (CTE) programs This line item counts the number of times clients have been referred to a Workforce Development/Career and Technical Education (CTE) subsequent to a counseling session. This line item only draws from the counseling session referral field, not the client field.

Goal 7: Capture Market Data in Key Industries

This section of the scorecard identifies those clients that are working on critical and emerging technologies. This line item counts all clients in the database that have indicated that they are working in one of the Key Technology Areas identified by DoD, and who have had counseling and/or training activity during the period.

The key technology areas are:

  • Advanced and Networked Sensors, Sensing, and Signature
  • Advanced Manufacturing
  • Advanced Materials
  • Artificial Intelligence (AI)
  • Biotechnologies
  • Communication and Networking Technologies
  • Computing and Software (Advanced)
  • Cyber and Integrated Sensing
  • Directed Energy
  • Financial Technologies
  • Future Generation Wireless Technology (FutureG)
  • Gas Turbine Engine Technologies (Advanced)
  • Human-Machine Interfaces
  • Hypersonics
  • Integrated Network Systems-of-Systems
  • Microelectronics and Semiconductors
  • Nuclear Energy Technologies (Advanced)
  • Quantum Science including Quantum Information Technologies
  • Renewable Energy Generation and Storage
  • Robotics and Autonomous Systems
  • Space Technologies and Systems

NOTE: The P6 Export file does not include clients that were only trained (i.e. you never "met" with them). Since the P6 Export file asks for the "Meeting Date" of the client who is working on critical and emerging technologies, OutreachSystems interpreted this more narrowly. If you want training clients to be included on the list, then you can manually add them based upon the scorecard list.

Line items not tracked natively in Neoserra

There are a number of goals identified by DoD that are not tracked natively in Neoserra and will need to be tracked separately in a spreadsheet. You can either create a separate Excel spreadsheet or you can use the Neoserra APEX Accelerator Metrics Log. It should be noted that this log is simply a place to record the data. You can use the log to record data that is tracked external to Neoserra. The benefit of using the Neoserra log is that these goals can be included on the scorecard.

  • Goal 1: Number of clients attempting to re-enter the DIB and GIB (e.g., the number that submitted bids).
  • Goal 2: Number of direct referrals and businesses resulted in success.
  • Goal 3: Number of FOCI publications provided such as newsletters, articles, and any press releases.
  • Goal 3: Number of clicks on online FOCI publications.
  • Goal 4: Number of cybersecurity publications provided such as newsletters, articles, and any press releases.
  • Goal 4: Number of clicks on online cybersecurity publications.
  • Goal 5: Number of proposals submitted to GIPs by clients who received technical assistance
  • Goal 6: Number of clients submitting white papers to Funding Opportunity Announcements (FOAs) as a direct result of receiving technical assistance
  • Goal 6: Number of clients submitting full proposals to FOAs as a direct result of receiving technical assistance

While Neoserra offers the Neoserra APEX Accelerator Metrics Log to track some of these external metrics, it should be noted that this module does not support the Processing Mode: "Include audit information."



Want more? Browse our extensive list of Neoserra FAQs.