12 KiB
OpenPAI Release Note
July 2020 (version 1.1.1)
July 2020 (version 1.1.0)
- Storage:
- Support readonly storage. (#4523)
- Security
- If ssl is enabled, all requests will use https. (#4550)
- Authentication
- Support nested AD group in AAD Mode. (#4639)
- Marketplace
- Integrate with new version of PAI marketplace.
- Others
- Add stress test for PAI API. (#4665)
- Resolve job always retry for port conflict. (#4384)
- Webportal/VScode use JS SDK + SDK improvement. (#4660)
- Align webportal submit default value with backend. (#4682)
- Document enhance. (#4700)
- Fix tensorboard v2 the logdir is not correct issue.
- Fix webPortal submit job help link broken.
- Fix ssh barrier bug.
For more details about this release, please refer to detailed release note.
May 2020 (version 1.0.1)
May 2020 (version 1.0.0)
With the v1.0.0 release, OpenPAI is officially switching to pure Kubernetes-based architecture. In addition to this, we had also made efforts on making our component design more modularized by re-organized the code structure to 1 main repo together with 7 standalone key component repos.
Please refer to the system architecture documentation for more detailed design thinkings about this change, and review the following list to get a better understanding about the 7 new component repos:
-
hivedscheduler is a new OpenPAI component providing various advantages over standard k8s scheduler, such as resource isolation for multiple tenants, GPU topology guarantee for virtual clusters, and better topology-aware gang scheduling with no resource starvation.
-
frameworkcontroller is built to orchestrate all kinds of applications on Kubernetes by a single controller.
-
openpai-protocol is the specification of OpenPAI job protocol. It facilitates platform interoperability and job portability. A job described by the protocol can run on different clusters managed by OpenPAI. The protocol also enables great flexibility. Any AI workload can be described by it conveniently.
-
openpai-runtime provides runtime support which is necessary for the OpenPAI protocol. OpenPAI runtime can classify typical runtime error patterns and prevent unnecessay error retries. Therefore cluster resource can be saved.
-
openpaisdk is a JavaScript SDK designed to facilitate the developers of OpenPAI to offer more user-friendly experience.
-
openpaimarketplace is a service which stores examples and job templates. Users can use it from webportal plugin to share their jobs or run-and-learn others' sharing job.
Features:
- Provide a way for team collaboration among pai users.
- Provide an easy-to-start and education for new users. Users could refer to shared templates in marketplace and learn how to use pai platform correct.
- Provide admin review process to ensure the quality of templates in marketplace.
-
openpaivscode is a VSCode extension, which makes users connect OpenPAI clusters, submit AI jobs, simulate jobs locally and manage files in VSCode easily.
The version of each standalone repo used in OpenPAI v1.0.0 is hivedscheduler v0.3.2, frameworkcontroller v0.6.0, openpai-protocol v2.0.0-alpha, openpai-runtime v0.1.0, openpaisdk v0.1.0, openpaimarketplace v1.2.0 and openpaivscode v0.3.0.
Other major new features and improvements come with this new release are:
- Based on kubespray, we provided a quick start script for you to deploy OpenPAI from scratch faster.
- In addition to basic authentication, support for Azure Active Directory (AAD) was added to provide SSO and multi-factor authentication for users.
- Built-in storage were refactored to support more storage types with a unified interface.
- Storage manager is provided for users to set up NFS+SMB storage server easily.
- Webportal UX is upgraded to Microsoft Fluent UI 7.0. UI component and style elements are optimized to provide more user-friendly experience for both user and admin scenarios.
- RESTful API has been refined to be cleaner and more well-organized. A detailed swagger document is also provided.
- In addition to current documentation, we provide an end-to-end usage manual for cluster users and administrators to learn how to use OpenPAI.
For more details about this release, please refer to detailed release note.
July 2019 (version 0.14.0)
Welcome to the July 2019 release of OpenPAI. There are a number of updates in this version that we hope you will like, some of the key highlights include:
- New webportal job submission experience - Update submit job UI to version 2.
- Python sdk of openpai is now ready! - You can config, submit and debug your job easily with python sdk.
- New yarn schedular to improve resource efficiency - Admin can bind dedicated Virtual Cluster to 1 or more physical nodes.
- vscode extension now supports submitting v2 job.
- Provide team storage plugin to manage data shared by team.
- How to upgrade to OpenPAI v-0.14.0?
For more details about this release, please refer to detailed release note.
June 2019 (version 0.13.0)
- OpenPAI protocol:
- Introduce OpenPAI protocol and job submission v2 (#2260)
- Add new job submission v2 plugin (#2461)
- Web portal:
- Add login page for guests (#2544)
- Add user home page (#2614)
- Add new user management page (#2726, #2796)
- User Management UX refactoring with new layout and themes (#2726, #2796)
For more details about this release, please refer to detailed release note.
April 2019 (version 0.12.0)
- Web portal:
- Display error message in job detail page #2456
- Import users from CSV file directly and show the final results #2495
- Add TotalGpuCount and TotalTaskCount into job list #2499
- Deployment
- Add cluster version info #2528
- Check if the nodes are ubuntu 16.04 #2520
- Check duplicate hostname #2403
For more details about this release, please refer to detailed release note.
April 2019 (version 0.11.0)
- Support team wise NFS storage, including:
- An NFS configuration plug-in and a commandline tool. #2346
- A simple NFS-job submit plug-in. #2358
- Refer to Simplified Job Submission for OpenPAI + NFS deployment for more details.
- New alerts for unhealthy GPUs, currently including following alerts #2209:
- Admin could know all running jobs on a node. #2197
- Filter supports in Job List View. #302
- Hold the Env for failed jobs which are caused by user error. #2272
For more details about this release, please refer to detailed release note.
Mar 2019 (version 0.10.1)
- Admin can configure MaxCapacity through REST API for a given Virtual Cluster so that the virtual cluster can use iddle resources as bonus. #2147
- Support Azure RDMA. #2091; how-to doc
- New Disk Cleaner for abnormal disk usage: The disk cleaner will check disk usage every 60 second(configurable), and if the disk usage is above 94%(configurable), it will kill the container that uses largest disk space using specific signal(10), the container will exit with code 1, and the related job will fail. Admin/User can track the reason in job logs. #2119
- Web portal: add "My jobs" filter button. #2111
- "Submit Simple Job" web portal plugin. #2131
For more details about this release, please refer to detailed release note.
Feb 2019 (version 0.9.1)
Feb 2019 (version 0.9.0)
- Add pai service dashboard to grafana, cluster admin can get pai services resource consumption from paiServiceMetrics page. #1694
- Support to add custom web pages to the web portal of PAI deployments with WebPortal Plugin, refer to Plugins Doc for how to use the new feature, and refer to PR 1700 for how PAI Marketplace is using it as an example.
- Support update virtual cluster dynamically from webportal. Please refer to virtual cluster management for how to use this new feature. #1831 #1974
- Support customized job environment variables. #1544
- Add VS Code client for PAI, please refer to OpenPAI VS Code Client for more detail.
For more details about this release, please refer to detailed release note.
Dec 2018 (version 0.8.3)
Nov 2018 (version 0.8.2)
Nov 2018 (version 0.8.1)
Oct 2018 (version 0.8.0)
- All user submitted jobs can be cloned and resubmitted in Job detail page #1448.
- The new designed Marketplace and Submit Job V2 are under public review. Please refer to the instruction for more information Marketplace and Submit job v2. Any feedback and suggestions are appreciated.
- Alerting service supports to mute alerts. The instructions can be found via alert-manager.
- New Feedback Button: users are allowed to submit GitHub Issues with appended OpenPAI version directly from WebUI #1289.
For more details about this release, please refer to detailed release note.
Sep 2018 (version 0.7.2)
Aug 2018 (version 0.7.1)
- Administrators can receive email notifications on cluster problems after set up the new supported "Alert Manager". Please read more about how to set up Alert Manager and the notification Rules.
For more details about this release, please refer to detailed release note.
July 2018 (version 0.6.1)
- The 'paictl' tool: Introducing paictl, the deployment/management tool with the functionalities of image building, service start/stop, k8s bootup/clean, and configuration generation.
- Single-box deployment: Support single-box deployment for evaluation purpose.
- New UI for user management: Now the console for administrators to manage PAI users has got a new UI.
- Documentation: Significant changes on documents -- more comprehensive, more structured, and easier to follow.
For more details about this release, please refer to detailed release note.