16453d6090
Bumps [pydantic](https://github.com/pydantic/pydantic) from 0.22 to 1.10.13. - [Release notes](https://github.com/pydantic/pydantic/releases) - [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md) - [Commits](https://github.com/pydantic/pydantic/compare/v0.22...v1.10.13) --- updated-dependencies: - dependency-name: pydantic dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> |
||
|---|---|---|
| data | ||
| deployment | ||
| docs | ||
| extract_skills | ||
| get_skill | ||
| services | ||
| tests | ||
| .dockerignore | ||
| .funcignore | ||
| .gitignore | ||
| Dockerfile | ||
| LICENSE | ||
| README.md | ||
| SECURITY.md | ||
| azure-pipelines.yml | ||
| extensions.csproj | ||
| host.json | ||
| requirements.txt | ||
README.md
Deprecation Notice
We've launched a better version of this service with Azure Cognitive Serivces - Text Analytics in the new V3 of the Named Entity Recognition (NER) endpoint.
NER V3 Information
You can read more about this work and how to use it here:
(NEW) Custom Entity Lookup Skill
Azure Cognitive Search recently introduced a new built-in Cognitive Skill that does essentially what this repository does. You provide a dictionary of terms you want to match and it will extract those for you from any text field in your search index. You can read more about that here: https://docs.microsoft.com/en-us/azure/search/cognitive-search-skill-custom-entity-lookup
Custom Cognitive Skill with Containers
This repo is no longer supported but you're free to use the index and skill definitions provided to enable the personalized job recommendations scenario. If you are just looking to deploy a container as a custom skill, I highly recommend utilizing this more generic cookiecutter repository: https://github.com/microsoft/cookiecutter-spacy-fastapi. We will continue to support this project. For more information on deploying Containers on Azure see:
Introduction
The Skills Extractor is a Named Entity Recognition (NER) model that takes text as input, extracts skill entities from that text, then matches these skills to a knowledge base (in this sample a simple JSON file) containing metadata on each skill. It then returns a flat list of the skills identified.
Definitions
What is a Cognitive Skill?
A Cognitive Skill is a Feature of Azure Search designed to Augment data in a search index.
What is a Skill in terms of the Skills Extractor?
A Skill is a Technical Concept/Tool or a Business related/Personal attribute.
Example skills: Machine Learning, Artificial Intelligence, PyTorch, Business, Advertising
For the current goals of the service, we are focused on technical skills. Technical skills are the abilities and knowledge needed to perform specific tasks. They are practical, and often relate to mechanical, information technology, mathematical, or scientific tasks. The Taxonomies the API pulls from primarily consist of concepts and tools related to technology. For example, Programming Languages are considered a higher-level technical skill, and C# or Python are a sub of that larger skill.
P.S. Sorry for the confusing naming.
Skill Sources
We pull skills and technologies from many open online sources and build Record Linkage models to conflate skills and categories across each source into a single Knowledge Graph.
Here is a list of our sources:
- Coursera
- Microsoft Academic Graph
- GitHub Featured Topics
- StackShare Tools
- Class Central Subjects Github
- ONET Online Hot Technology Index
- ACM Classifications
Use Cases
The original idea stemmed from a few organizational needs. Here are a few:
-
Determine the skills required for a job opening at your company and match applicant resumes based on skills.
-
Extract skills from Learning Content that your company creates to improve search and recommendations.
-
Identify the technical and professional skills of your team or organization and work to close skill gaps.
Prerequisites
Before running this sample, you must have the following:
- Install the Azure CLI. This article requires the Azure CLI version 2.0 or later. Run
az --versionto find the version you have.
You can also use the Azure Cloud Shell.
Dependencies
- SpaCy v2.1 - NLP in Python https://spacy.io
- Azure Functions Python Worker - https://github.com/Azure/azure-functions-python-worker
Quickstart: Extract Skills for your data in Azure Search using a Custom Cognitive Skill
If you're unfamiliar with Azure Search Cognitive Skills you can read more about them here: https://docs.microsoft.com/en-us/azure/search/cognitive-search-concept-intro
Follow one of the scenarios below
- Extract Skills from an Existing Search Index
- Use the sample Search Scenario of extracting Skills from Jobs and Resumes
Create your own Custom Cognitive Skill
If you would like to create your own Custom Skill leveraging the NLP power of the Python Ecosystem you can use this cookiecutter project to bootstrap a containerized API to deploy in your own infrastructure.
https://github.com/Microsoft/cookiecutter-azure-search-cognitive-skill
Feedback & Support
We’re eager to improve, so please take a couple of minutes to answer some questions about your experience https://aka.ms/AA4xoy5. For support, please contact: WWL_Skills_Service@microsoft.com
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.