/
Test Evaluation Guide

Test Evaluation Guide

Owned by Tim Brooks
Last updated: Mar 09, 2020 by Petr PlavjaníkVersion comment

The Zowe Conformance Test Evaluation Guide is a set of self-certify and self-service tests to help the developer community integrate and extend specific technology into the Zowe framework. 


Below are the requirements for the available conformance programs. Items marked (required) are required for achieving conformance in a given program.

 

Zowe API Mediation layer - LTS v1 Conformance Criteria

  • Application Service
    1. An application provides at least one service or UI (required)
  • Register with discovery services
    1. A service must be registered using one of the following methods: (required)

Dynamic registration - preferred (best practice)

Static definition - minimum requirement

  1. The service must provide a default service ID that is prefixed by the provider name (for example: `acme`, `xyzcorp`, `bar`). (required)
  2. The service ID must be configurable externally after deployment. (required)
  3. The service ID must written in lower case, contain no symbols, and is limited to 64 characters. (required)
  4. The API ID must follow the same rules as for Java packages. The example of the API ID: zowe.apiml.apicatalog. (required)
  5. The published service URL must follow the gateway URL conventions. (required)
  6. For versioned APIs, service URLs must contain a service version before the service ID in the following formats:
           - api/v1/{serviceId} reserved for REST APIs
             - ui/v1/{serviceId} reserved for UIs
             - ws/v1/{serviceId} reserved for WebSockets
    For non-versioned APIs or APIs versioned differently (e.g. z/OSMF), use the following formats: 
             - api/{serviceId} reserved for REST APIs
             - ui/{serviceId} reserved for UIs
             - ws/{serviceId} reserved for WebSockets
  7. The registration of the service must not be done by modifying the Zowe runtime directory api-defs folder (required).  Supported methods are:
    1. adding the static API definition YAML file path to instance.env file for the Zowe workspace.  
    2. copying the static API definition YAML file to the instance directory workspace api-definitions directory 
  • API Documentation
    1. Documentation is Swagger/Open API 2.0/Open API 3.0 compliant. (required)
    2. Every public resource is documented with a description of each resource. (required)
    3. Every method of each REST endpoint is documented. (required)
    4. Every method of each REST endpoint is demonstrated by an example. (required)
    5. Every parameter (headers, query parameters, payload, cookies, etc.) is documented with definitions of all possible values and their associated meanings. (required)
    6. Every HTTP error code must be documented. If endpoint has additional more granular error codes just the documentation reference can be provided for these. (required)
  • API naming and addressing
    1. Encoded slash is not used. - preferred (best practice)
    2. The service interprets values independent of their URL encoding. (required)
    3. lowerCamelCase is used for names of resources, parameters, and JSON properties.
  • Service requests and responses
    1. API - Request and response payloads are in JSON or binary data format. - preferred (best practice)
    2. API - In JSON format, links are relative, and must not contain the schema, hostname, and port. (required)
    3. WebSocket - Service URIs contained in WebSocket messages payload are addressed through the API ML Gateway. (required)
    4. UI - UI uses relative links and does not contain the schema, hostname, and port. (required)
  • Authentication and Authorization
    1. Resources are protected by mainframe credentials. (required)
    2. Services accept basic authentication (minimum requirement). (required)
    3. In order to support Zowe Single-Sign-On, services have to accept Zowe JWT token in the cookie, or support PassTickets. - preferred (best practice)
  • Versioning and Support
    1. Service implementation follows the semantic versioning model.
    2. Last two major versions are supported by API services. (required)
  • UI
    1. UI uses only relative URLs. (required)
  • WebSocket Services
    1. WebSocket connection creation, all subsequent communication between WebSocket client, and server is routed through the API ML Gateway. (required)
    2. WebSocket connections are closed by the initiator through API ML Gateway. (required)

Lifecycling - optional

  • Running as a Zowe address space
    1. If the service should be lifecycled by Zowe then it should provide a fully qualified path in the instance.env file for the Zowe workspace which points to the location of a directory containing a start.sh script (required) , a validate.sh script (optional) and a configure.sh script (optional).  
    2. If the service introduces new variables to the instance.env file, these should be prefixed by the provider ID to avoid collisions (required).  

Directory and file ownership and permissions - required

  • Directory modification
    1. A conformant application must not modify the contents of the Zowe runtime USS directory and it must not change any directory or file permissions or ownership within the Zowe runtime (required).
    2. A conformant application must not modify the permissions or ownership of a Zowe instance directory workspace.  

                                                                                                                    

Zowe App Framework – 2020

  • Packaging
    1. Every plugin must have a unique ID. The ID format follows java package naming conventions. The Zowe project reserves org.zowe. (required)
    2. Every plugin and each of its services must have a version. (required)
    3. Directory layout adheres to the App filesystem structure. (required)
    4. Source code is also recommended, but not required to adhere to the App filesystem structure for tooling consistency.
  • Web UIs ALL
    1. All Apps must contain an icon image file to represent it, located at web/assets/icon.png within the App's package. (required)
  • Web UI IFrame
    1. IFrame Apps (apps with framework type "iframe") which embed a top-level iframe within (example: https://github.com/zowe/api-layer/blob/master/zlux-api-catalog/web/index.html) must use the ID "zluxIframe" for that element. This is required for the app to be a recipient of app to app communication. (required)
    2. Zowe resources must be accessed via the iframe-adapter located within zlux-app-manager/bootstrap/web.  Use of window.parent or window.top to access the ZoweZLUX object is non-permissible. (required)
    3. Documentation or automated addition of the "iframe" plugin to the Zowe desktop must be performed by executing the script 'zowe-install-app.sh' script in the Zowe instance directory.  (required)
  • Web UI Non-IFrame
    1. DOM elements originating from your App should always be a child of the Zowe viewport DOM element, "com-rs-mvd-viewport". (required)
    2. Network requests to the Zowe App Server must never be done without the use of the URI Broker. (required)
    3. Access to resources outside the App Server should also be made through the URI Broker whenever possible.
    4. Apps must not pollute the global namespace with regards Javascript, HTML, and CSS. (required)
    5. When using a library present in the Zowe App Framework core, you must depend on the same version. (required)
    6. Web apps should extend the framework's default build scripts for webpack and typescript.
  • UI Design
    1. Apps should follow the UI Design guidelines at https://github.com/zowe/zlc/blob/master/process/UI_GUIDELINES.md
  • Localization and Internationalization (I10n and I18n)
    1. The active language to be used for string selection must be retrieved using ZoweZLUX.globalization.getLanguage(), which determines language by multiple factors. (required)
    2. No strings visible in a UI should be hard-coded, rather resource strings must be used in accordance with one of the existing internationalization support mechanisms.
  • App Server
    1. Data services should be written such that all synchronous and asynchronous errors are caught. Utilize try-catch and check the existence of error objects from asynchronous calls. Uncaught exceptions effect server responsiveness and disrupt clients. (required)
  • Documentation
    1. Every HTTP API must be documented in swagger 2.0. The swagger document must be stored in doc/swagger. (required)
    2. In addition, it is recommended to have documentation about the format of any Websocket APIs, to be placed within doc.
  • Logging
    1. An Apps non-IFrame web components, or App Framework dataservices (eg Javascript and Typescript) must log only through the "zlux" logger. (required)
    2. ZSS services log only through the Zowe ZSS Logger. (required)
    3. Passwords must never be logged. (required)
    4. Error reporting should follow the standard tooling.
  • Encoding
    1. If you want your Apps to work with z/OS Node.js version 12 or greater, all application files must be tagged according to their content type. (required)
    2. Testing Apps via the install-app script is advisable to allow end users to utilize Zowe plugin management tooling.
  • Storage
    1. User preferences, if applicable to a plugin, must be stored through the configuration data service. (required)
    2. For other plugin storage needs, storing data outside of the configuration dataservice is permitted only within $INSTANCE_DIR/workspace/app-server or $INSTANCE_DIR/workspace/app-server/pluginStatic with a top-level folder equal to their plugin ID. Plugins must not store information anywhere else in any Zowe directories such as $INSTANCE_DIR or $ROOT_DIR in order to prevent conflict with future Zowe versions and other plugins. (required)
    3. It is advisable for the storage of user preferences to use environment variables for locating directories.  Use of the instance directory environment variable is not required, but should be considered to subvert the use of hard-coded paths.

  • Directory and file ownership and permissions - required

    1. A conformant application must not modify the contents of the Zowe runtime USS directory and it must not change any directory or file permissions or ownership (required).
    2. A conformant application must not modify the permissions or ownership of a Zowe instance directory workspace.  

  • Lifecycling as a Zowe address space - optional

    • If the service should be lifecycled by Zowe then it should provide a fully qualified path in the instance.env file for the Zowe workspace which points to the location of a directory containing a start.sh script (required) , a validate.sh script (optional) and a configure.sh script (optional).  
    • If the service introduces new variables to the instance.env file, these should be prefixed by the provider ID to avoid collisions (required).  


-----------------------------------------------------------------------------------------------------------------------------------------------------------------------


Zowe CLI - LTS v1 Conformance Criteria 


  • Infrastructure
    1. Plug-in is constructed on the Imperative CLI Framework. (required)
    2. Plug-in is NOT run as a standalone CLI. (required)
    3. Plug-in commands write to stdout or stderr via Imperative Framework response.console APIs. (required)
  • Installation
    1. Plug-in is installable with the zowe plugins install command. (required)
    2. Plug-in is installable into the @zowe-v1-lts version of the core Zowe CLI and follows semantic versioning. (required)
    3. Plug-in is uninstallable via the zowe plugins uninstall command. (required)
  • Naming
    1. If the plug-in introduces a command group name, it does not conflict with existing conformant plug-in group names. (required)
  • Profiles
    1. If the plug-in has unique connection details, it introduces a profile that lets users store these details for repeated use. (required)
    2. Plug-in users are able to override all profile settings via the command line and/or environment variables.



Related content