Required Testing Information for the AppExchange (ISV) Security Review

Most offerings contain one or more parts that are classified as Native, Composite, or Client/Mobile. Our approach is to test all parts of the offering to ensure that our mutual customers and their data are not put at risk. (See Scope

The sections below describe the information that the security review team will need in order to test your offering. It’s very important to provide all of the required information for testing. If the security review team does not have the full end-to-end test environment and the other information below significant delays can be introduced. If you are unsure if part of your offering is in scope for our testing please include it anyway. The review team will not test parts of the app that are out of scope based on our testing criteria.

1. Reports from security testing tools that are free from issues (See Requirements Checklist)

Depending on the composition of your offering you will need to provide one or more of the following reports: 

  • Force.com Security Code Scanner (Checkmarx) report for your Apex and Visualforce Code run on your development or packaging organization (org). (Required if the offering includes Apex or Visualforce) 
  • Web Application Scanner reports (Burp) for any integrated external web applications or web services. (Required if there is an external web application or web service) 
  • Additional reports for any applicable specialized tools that were used to test your application. (e.g. SWFscan, Nessus, Nikto, SSLScan, Nmap)
2. A dedicated fully configured end-to-end test environment

Note: It’s important to provide credentials and access for all parts of the offering. 

Offerings that include a native (Apex, Visualforce or Lighning) part: 
  • A dedicated fully configured Salesforce testing org including Salesforce Login and password for all user levels (admin, end user, approver etc.) with the latest released version of the package that you intend to distribute installed. If sample data is required for the application to function please include a logical set of sample data. Please also include any API endpoint URL(s) external to Salesforce accessed by the package. If you do not have a dedicated test org you can sign up for a Free Developer Account
Starting March 1, 2017 full Locker Compliance will be mandatory to submit for the security review. Therefore, the Developer Edition test environment provided for the security review must have Locker Service enabled. Enterprise edition and sandbox orgs will not be acceptable test environments starting March 1, 2017.

If the offering is a single org or hybrid single org solution provide detailed documentation of the sharing/access control implementation. The review team recommends creating a free developer account because accounts do not expire and are provided at no cost. These will be the only acceptable test environments starting March 1, 2017. Keep in mind that the review team does not need or want direct access to the org where you have developed or packaged your application. The test org should be a separate dedicated org. Please note that the review team cannot access accounts where support access is granted. That functionality is not available to our testers, and does not provide the correct level of access. 

Offerings that include an external web application or web service (composite app): 
  • Login URL(s) and credentials for all user levels (admin, end user, approver etc.) for the application that is integrated in or optional to your offering. If sample data is required for the application to function please include a logical set of sample data. 
  • Salesforce credentials that are associated with the offering. Please see section above “Offerings that include a native (Apex and Visualforce) part:” 
Offerings that include a client application:
  • A fully configured Virtual machine for the Client application including any required documentation. If credentials other than the Salesforce account login, or related external application credentials are required or optional for the client application please provide them as well. If sample data is required for the application to function please include a logical set of sample data. 
  • Salesforce credentials that are associated with the offering. Please see section above “Offerings that include a native (Apex and Visualforce) part:” 
OR 
  • The installer for the Client application. Please provide the installer or a download for the Client application, and any required license files, associated sample data, config guides etc. If credentials other than the Salesforce account login, or related external application credentials are required or optional for the client application please provide them as well. If sample data is required for the application to function please include a logical set of sample data. 
  • Salesforce credentials that are associated with the offering. Please see section above “Offerings that include a native (Apex and Visualforce) part:” 
Offerings that contain a Mobile Application:
  • The iOS Mobile application. Please provide the install link if the application is free and already published to the Appstore. If the application is not yet approved or is not free, please either provide an ad-hoc installation (contact us for device UDIDs), or a Testflight link for the app. (no UDID required) More information about Testflight is available at:

    https://testflightapp.com/ 

  • The Android Application. Please provide the .APK for the android application and the target device. 
  • The Blackberry Application. Please provide the .COR for the Blackberry application and the target device. 
  • The Windows Phone Application. Please provide the .XAP for the application and the target device. If credentials other than the Salesforce account login, or related external application credentials are required or optional for the mobile application please provide them as well. If sample data is required for the application to function please include a logical set of sample data. 
  • Salesforce credentials that are associated with the offering. Please see section above “Offerings that include a native (Apex and Visualforce) part:”
3. Documentation
  • A description of the offering and any notable information 
  • Technical point of contact for the review (Name and Email) 
  • Describe all integration points within the entire data flow of the offering including Salesforce, external web applications and client/mobile components. Also please note the method of authentication/authorization used between the integration points. 
  • Describe what Salesforce data (custom/standard objects, custom fields, credentials, etc) is sent/received/accessed/processed/modified/stored etc the web applications, web services or client/mobile applications external to Salesforce. Please explain the end-to-end data flow and external storage of this data. 
  • Provide application documentation with use-cases if available 
  • Describe Role and access-level descriptions if separate role-based access control is applicable 
  • Describe technologies used (application framework, programming languages, client-side technologies (Flash, Silverlight etc.), database type) 
  • If your organization has a documented information security policy please provide the policy and a valid compliance point of contact. Sample Policy Template - Here are some sample policy templates to guide you in creating your company security and operational policies.