Edit complete
This commit is contained in:
Linda Caputo
GitHub
Родитель
d4c7230713
Коммит
727668260f
|
|
@ -17,7 +17,16 @@ geolocation: [Asia]
|
|||
|
||||
Nexon delivers online game entertainment and is a market leader in providing fun and deeply immersive content to a worldwide audience. In a hackfest with Microsoft, Nexon worked on ways to improve artificial intelligence to make the game experience more captivating for players.
|
||||
|
||||
The hack team included the following:
|
||||
### Key technologies used
|
||||
|
||||
- [Azure Machine Learning](https://azure.microsoft.com/en-us/services/machine-learning/)
|
||||
- [Azure Functions](https://azure.microsoft.com/en-us/services/functions/)
|
||||
- [Web Apps feature of Azure App Service](https://azure.microsoft.com/en-us/services/app-service/web/)
|
||||
- [Azure Blob storage](https://azure.microsoft.com/en-us/services/storage/blobs/)
|
||||
- [Azure Table storage](https://azure.microsoft.com/en-us/services/storage/tables/)
|
||||
- [Azure SQL Database](https://azure.microsoft.com/en-us/services/sql-database/?v=16.50)
|
||||
|
||||
### Core team
|
||||
|
||||
- Kitae Noh – G5, Nexon Korea
|
||||
- Jaesuck Kim – E1, Nexon Korea
|
||||
|
|
@ -35,47 +44,50 @@ The hack team included the following:
|
|||
- Eunji Kim – Technical Evangelist, Microsoft
|
||||
|
||||
|
||||
## Customer profile ##
|
||||
## Customer profile
|
||||
|
||||
|
||||
|
||||
|
||||
[Nexon Korea Corporation](http://www.nexon.net/) is the No. 1 game company in Korea in terms of revenue. Established in 1994, Nexon is a pioneer and global leader in the MMORPG (Massive Multiplayer Online Role Playing Game) industry. It has grown into an international company with the establishment of several branches: Nexon Japan, Nexon America, and Nexon Europe. It is aggressively establishing a presence in the global market with high-quality games and services, serving 1.4 billion users in more than 110 countries.
|
||||
|
||||
Nexon has [developed and published](http://www.nexon.com/Home/Game.aspx) more than 50 world-class PC online games and 30 mobile games in a series and is currently developing 20+ mobile projects with heavy financial investment.
|
||||
Nexon has developed and published more than [50 world-class PC online games and 30 mobile games](http://www.nexon.com/Home/Game.aspx) in a series and is currently developing 20+ mobile projects with heavy financial investment.
|
||||
|
||||
|
||||
## Pain point or problem area to be addressed ##
|
||||
## Problem statement
|
||||
|
||||
Thanks to the success of the [Mabinogi](http://mabinogi.nexon.net/landing/) MMORPG, Nexon published a [trading card game (TCG)](https://en.wikipedia.org/wiki/Collectible_card_game) with Mabinogi intellectual property named *Mabinogi Duel*, available from [Google Play](https://play.google.com/store/apps/details?id=com.nexon.devcat.mabinogiduel.global) and the [App Store](https://itunes.apple.com/us/app/mabinogi-duel/id1033398935?mt=8).
|
||||
|
||||
For a TCG, artificial intelligence (AI) is the most important part when it comes to player versus environment (PvE) play. Most of the game's players leave the play queue because the AI is no fun and the typical play pattern makes the game boring. Nexon needs refined solutions to improve the game AI, such as scalable real-time web service and batch analytics to reduce the customer churn rate and to increase the profit.
|
||||
|
||||
## Solution, steps, and delivery ##
|
||||
## Solution, steps, and delivery
|
||||
|
||||
Nexon needed to improve the game AI and scalable real-time web service, such as game card deck suggestions and batch tasks to predict customer churn rate.
|
||||
|
||||
So, we proposed Azure Machine Learning analytics and the Web Apps feature of Azure App Service as a complex gameplay front-end RESTful API, which helps to implement authentication and massive Azure Machine Learning API call for a scalable service. We also proposed Azure Functions as a batch analytics task scheduler processor.
|
||||
So, we proposed Microsoft Azure Machine Learning analytics and the Web Apps feature of Azure App Service as a complex gameplay frontend RESTful API, which helps to implement authentication and massive Azure Machine Learning API call for a scalable service. We also proposed Azure Functions as a batch analytics task scheduler processor.
|
||||
|
||||
|
||||
|
||||
<br/>
|
||||
|
||||
The Nexon Mabinogi Duel team and Microsoft created an Azure-based solution architecture. We were able to achieve real-time analytics and schedule batch analytics results by using Web Apps and Azure Functions.
|
||||
|
||||
### Real-time predicted analytics with Web Apps
|
||||
|
||||
To implement real-time analytics, the hackfest team adopted flexible Web Apps for the front end because the Nexon mobile dev team was using C#, Node.js, and other languages and frameworks. Furthermore, since auto-scale and easy deployment (continuous deployment) are the most necessary features for the Nexon mobile dev team, Web Apps seemed to be the best choice.
|
||||
To implement real-time analytics, the hackfest team adopted flexible Web Apps for the frontend because the Nexon mobile dev team was using C#, Node.js, and other languages and frameworks. Furthermore, because auto-scale and easy deployment (continuous deployment) are the most necessary features for the Nexon mobile dev team, Web Apps seemed to be the best choice.
|
||||
|
||||
|
||||
|
||||
<br/>
|
||||
|
||||
GitHub repository: [https://github.com/m-duel-project/frontend-realtime](https://github.com/m-duel-project/frontend-realtime)
|
||||
|
||||
The basic role of the front-end web app is separated from the on-premises LUA game server, receiving real-time analytics requests and handling requests for Azure Machine Learning predictive analytics.
|
||||
> Frontend realtime GitHub repository: [https://github.com/m-duel-project/frontend-realtime](https://github.com/m-duel-project/frontend-realtime)
|
||||
|
||||
This service controls the access and checks basic authentication and AES-encrypted JSON data that is processed as RESTful API service. This service was implemented with ASP.NET Web API during the hackfest. Web API performs the stages listed next.
|
||||
<br/>
|
||||
|
||||
##### Role of front-end web app
|
||||
The basic role of the frontend web app is separated from the on-premises LUA game server, receiving real-time analytics requests and handling requests for Azure Machine Learning predictive analytics.
|
||||
|
||||
This service controls the access and checks basic authentication and AES-encrypted JSON data that is processed as a RESTful API service. This service was implemented with ASP.NET Web API during the hackfest. Web API performs the stages listed next.
|
||||
|
||||
#### Role of frontend web app
|
||||
|
||||
- Request/response to Azure Machine Learning predictive analytics
|
||||
- Store predicted data on NoSQL – Azure Table storage
|
||||
|
|
@ -83,12 +95,16 @@ This service controls the access and checks basic authentication and AES-encrypt
|
|||
|
||||
|
||||
|
||||
<br/>
|
||||
|
||||
|
||||
|
||||
<br/>
|
||||
|
||||
> Azure Machine Learning model repository: [https://github.com/m-duel-project/azure-hands-on/blob/master/docs/azure_ml_tutorial.md](https://github.com/m-duel-project/azure-hands-on/blob/master/docs/azure_ml_tutorial.md)
|
||||
|
||||
<br/>
|
||||
|
||||
Azure Machine Learning model repository: [https://github.com/m-duel-project/azure-hands-on/blob/master/docs/azure_ml_tutorial.md](https://github.com/m-duel-project/azure-hands-on/blob/master/docs/azure_ml_tutorial.md)
|
||||
|
||||
We designed the service to receive real-time prediction requests from the gameplayer device, send it to Azure Machine Learning, store predicted results on Azure Table storage for log purposes, and lastly, save the predicted data on relational database management system (RDBMS) - Azure SQL Database.
|
||||
|
||||
Azure Table storage is key/value-based NoSQL storage used not just for C# but also for Node.js and other languages as easy proxy APIs. Azure Table storage was the best option for log analytics and log data storage, as it is persistent and stable.
|
||||
|
|
@ -104,9 +120,11 @@ We went through the following procedures by using these development/testing tool
|
|||
- For Azure Table storage dev/testing purpose, use [Azure Storage Explorer Tool](http://storageexplorer.com/).
|
||||
- Save predicted results from Azure Machine Learning on Azure SQL Database.
|
||||
|
||||
Code repository link: [https://github.com/m-duel-project/frontend-realtime](https://github.com/m-duel-project/frontend-realtime)
|
||||
|
||||
The following is a Web API code snippet.
|
||||
> Frontend realtime code repository link: [https://github.com/m-duel-project/frontend-realtime](https://github.com/m-duel-project/frontend-realtime)
|
||||
|
||||
<br/>
|
||||
|
||||
*Web API code snippet*
|
||||
|
||||
```
|
||||
// GET api/values
|
||||
|
|
@ -126,7 +144,9 @@ public IEnumerable<string> Get()
|
|||
}
|
||||
```
|
||||
|
||||
Code - calling Azure Machine Learning.
|
||||
<br/>
|
||||
|
||||
*Code - calling Azure Machine Learning*
|
||||
|
||||
```
|
||||
public static string HttpPostRequestResponseService()
|
||||
|
|
@ -165,7 +185,9 @@ public static string HttpPostRequestResponseService()
|
|||
}
|
||||
```
|
||||
|
||||
Code - store predicted data from Azure Machine Learning to Azure Table storage.
|
||||
<br/>
|
||||
|
||||
*Code - store predicted data from Azure Machine Learning to Azure Table storage*
|
||||
|
||||
```
|
||||
public static void InsertEntity()
|
||||
|
|
@ -182,12 +204,15 @@ public static void InsertEntity()
|
|||
}
|
||||
```
|
||||
|
||||
<br/>
|
||||
|
||||
|
||||
|
||||
<br/>
|
||||
|
||||
While implementing real-time prediction requests, the hackfest team encountered the following error and fixed it accordingly.
|
||||
|
||||
##### Azure Machine Learning real-time request fail and debugging
|
||||
**Azure Machine Learning real-time request fail and debugging**
|
||||
|
||||
Code from ASP.NET Web API to Azure Machine Learning asynchronous code failed with an HTTP 400 error. The hackfest team turned on the Azure Machine Learning log, but had difficulty finding out a root cause. The team tried HttpWebRequest instead of HttpClient object, but received the same HTTP 400 error.
|
||||
|
||||
|
|
@ -198,6 +223,8 @@ Through the inspection, the team found that Korean text data columns were broken
|
|||
|
||||
|
||||
|
||||
<br/>
|
||||
|
||||
There are many ways to implement a scheduler task on Microsoft Azure—Azure Functions app, Azure WebJobs, Azure Scheduler, and Quartz framework on a virtual machine.
|
||||
|
||||
During the hackfest, the team reviewed scheduler services and chose the Azure Functions app for serverless coding, batch analytics blob, timer, and webhook trigger to enable the team to focus on the code itself.
|
||||
|
|
@ -206,16 +233,16 @@ We went through the following procedures:
|
|||
|
||||
- We assume that data to be analyzed by batch prediction is generated automatically or manually from the game admin website.
|
||||
- We assume that when the game admin executes batch analytics, a JSON format file will be uploaded on an Azure Blob storage "input" container, either on-demand or periodically.
|
||||
- Azure Functions app, the back-end processor, reads the file from Azure Blob storage on an upload trigger and can be executed by webhook, if necessary.
|
||||
- Azure Functions app, the backend processor, reads the file from Azure Blob storage on an upload trigger and can be executed by webhook, if necessary.
|
||||
- Azure Functions app receives the file on Blob storage via batch responses from Azure Machine Learning.
|
||||
- The results received from Azure Machine Learning are saved on Blob storage "output" container.
|
||||
- The results received from Azure Machine Learning are saved on a Blob storage "output" container.
|
||||
- By triggering of Blob storage "output," the Azure Functions app saves batch results to Azure SQL Database.
|
||||
|
||||
#### Azure Functions app code implementation
|
||||
|
||||
Code built with two Azure Functions apps and executed with BlobTrigger.
|
||||
The following code is built with two Azure Functions apps and executed with BlobTrigger.
|
||||
|
||||
- FirstTrigger: monitoring input path as BlobTrigger → build "batch job" and start with "job id"
|
||||
*FirstTrigger: monitoring input path as BlobTrigger → build "batch job" and start with "job id"*
|
||||
|
||||
```
|
||||
#r "Microsoft.WindowsAzure.Storage"
|
||||
|
|
@ -372,7 +399,9 @@ public static void Run(string myBlob, TraceWriter log)
|
|||
}
|
||||
```
|
||||
|
||||
- SecondTrigger: monitoring output path as BlobTrigger → Insert output blob to Azure SQL Database table
|
||||
<br/>
|
||||
|
||||
*SecondTrigger: monitoring output path as BlobTrigger → Insert output blob to Azure SQL Database table*
|
||||
|
||||
```
|
||||
#r "System.Data"
|
||||
|
|
@ -404,16 +433,19 @@ public static void Run(string myBlob, TraceWriter log)
|
|||
}
|
||||
```
|
||||
|
||||
<br/>
|
||||
|
||||
### Implement in-game Machine Learning
|
||||
|
||||
During the hackfest, our team implemented Game User Churn Prediction and Game User Item Suggestion with Azure Machine Learning. In addition, we also prototyped predicting analysis processes into the in-game client to successfully perform TCG card deck suggestion and improve the game AI pattern.
|
||||
|
||||
The hackfest team studied *unsupervised learning*, and tried most usable R algorithms. The objective was to call R function in C# and to test various R algorithms. To test such tasks, the hackfest team built an Iris data model and built in-game Iris data prediction code.
|
||||
The hackfest team studied *unsupervised learning*, and tried most usable R algorithms. The objective was to call the R function in C# and to test various R algorithms. To test such tasks, the hackfest team built an Iris data model and built in-game Iris data prediction code.
|
||||
|
||||
Iris data link repository: [https://github.com/m-duel-project/ml-data](https://github.com/m-duel-project/ml-data)
|
||||
> Iris data link repository: [https://github.com/m-duel-project/ml-data](https://github.com/m-duel-project/ml-data)
|
||||
|
||||
Code link repository: [https://github.com/m-duel-project/in-game-rl](https://github.com/m-duel-project/in-game-rl)
|
||||
> Code link repository: [https://github.com/m-duel-project/in-game-rl](https://github.com/m-duel-project/in-game-rl)
|
||||
|
||||
<br/>
|
||||
|
||||
```
|
||||
###################################
|
||||
|
|
@ -478,41 +510,50 @@ namespace ConsoleApplication1
|
|||
}
|
||||
```
|
||||
|
||||
They could execute R code and load RData function by using RDotNet project.
|
||||
<br/>
|
||||
|
||||
R.NET project: [https://www.nuget.org/packages/R.NET.Community/1.6.5](https://www.nuget.org/packages/R.NET.Community/1.6.5)
|
||||
They could execute R code and load the RData function by using the RDotNet project.
|
||||
|
||||
R code can be interop-called in C#, C++ or LUA. In this way, the Nexon Mabinogi team is building extensive knowledge about Machine Learning-related technologies and implementing advanced analytics code in the game.
|
||||
> R.NET project: [https://www.nuget.org/packages/R.NET.Community/1.6.5](https://www.nuget.org/packages/R.NET.Community/1.6.5)
|
||||
|
||||
<br/>
|
||||
|
||||
R code can be interop-called in C#, C++, or LUA. In this way, the Nexon Mabinogi team is building extensive knowledge about Machine Learning-related technologies and implementing advanced analytics code in the game.
|
||||
|
||||
Nexon’s issue is a comprehensive scenario, which most mobile game companies are encountering—to gain more revenue from their service with scalable and data scientific methods.
|
||||
|
||||
|
||||
|
||||
## Conclusion ##
|
||||
<br/>
|
||||
|
||||
## Conclusion
|
||||
|
||||
During the hackfest, Nexon developed real-time predicted analytics and batch predict analysis according to the solution architecture diagram.
|
||||
|
||||
Nexon originally planned to establish an AWS IaaS front end with AWS Machine Learning as a back end *or* AWS IaaS with [Tensorflow](https://www.tensorflow.org/), Python [Pandas libraries](http://pandas.pydata.org/), and [scikit-learn](http://scikit-learn.org/stable/index.html) machine learning toolkits.
|
||||
Nexon originally planned to establish an AWS IaaS frontend with Amazon Machine Learning as a backend *or* AWS IaaS with [Tensorflow](https://www.tensorflow.org/), Python [Pandas libraries](http://pandas.pydata.org/), and [scikit-learn](http://scikit-learn.org/stable/index.html) machine learning toolkits.
|
||||
|
||||
But by running the hackfest, Nexon successfully built a RESTful API front end with Web Apps with the added value of scalability and reduced management points, which keeps them focused on the code itself.
|
||||
But by running the hackfest, Nexon successfully built a RESTful API frontend with Web Apps with the added value of scalability, and reduced management points, which keeps them focused on the code itself.
|
||||
|
||||
Also, serverless, easy code—C# script and Node.js—was a "wow" factor for the Nexon hackfest developers because they could build an event trigger without a single line of code and just build the back-end batch analytics logic code on Azure Functions app.
|
||||
Also, serverless, easy code—C# script and Node.js—was a "wow" factor for the Nexon hackfest developers because they could build an event trigger without a single line of code and just build the backend batch analytics logic code on Azure Functions app.
|
||||
|
||||
> “Machine Learning is a hot issue in the mobile and online gaming industry and beyond. So Nexon DevCat HQ was prepared in various ways to adopt it. Through the Microsoft three-day hackfest, with a full utilization of Web Apps and Azure Machine Learning for Azure PaaS, we could develop a perfect prototype by concentrating on an immediate predictive analytics service.
|
||||
|
||||
> "The hackfest has been a great help for us to be able to build fundamental technologies for AI development, which makes smarter and more actions available. In the process, Machine Learning predictive analytics technologies were applied with counsel from a Microsoft partner, DS-eTrade. The Nexon, DS-eTrade, and Microsoft developers’ impassioned participation in the hackfest motivated everyone in the room. Since we believe a hackfest is the best way to collaborate and review new technologies, we plan to actively adopt it.”
|
||||
|
||||
> – Dong-Geon Kim, Director of Nexon DevCat HQ
|
||||
> —Dong-Geon Kim, Director of Nexon DevCat HQ
|
||||
|
||||
<br/>
|
||||
|
||||
|
||||
|
||||
<br/>
|
||||
|
||||
## General lessons ##
|
||||
### General lessons
|
||||
|
||||
With the Web Apps feature of Azure App Service, Nexon deployed a massive RESTful API front end that hosts various OSS languages and frameworks instantly. The Nexon developers were satisfied with the fast development focusing only on code itself. Also, they could build infrastructure not only on a previous IaaS virtual machine but on PaaS - App Service as well, with continuous deployment and staging environments during the hackfest.
|
||||
With the Web Apps feature of Azure App Service, Nexon deployed a massive RESTful API frontend that hosts various OSS languages and frameworks instantly. The Nexon developers were satisfied with the fast development focusing only on code itself. Also, they could build infrastructure not only on a previous IaaS virtual machine but on PaaS - App Service as well, with continuous deployment and staging environments during the hackfest.
|
||||
|
||||
Additionally, to accomplish massive batch analytics based on Azure Machine Learning, the developers had to build a batch trigger that could check the processing and updating status on legacy DBMS. With regard to that task, Nexon had to build their own scheduler process daemon with polling method. However, the Azure Functions app changed everything that triggers the timer, the blob container, and various other objects.
|
||||
|
||||
Since the Azure Functions app can trigger the blob container, Nexon just uploaded massive predictive analytics data on the blob container from their legacy admin tool. When the batch task ends, events could be triggered by Azure Functions for update status and legacy monitoring tools. Without any infrastructure building, polling process or trigger daemon process, they only need to write code with their preferred development language.
|
||||
Because the Azure Functions app can trigger the blob container, Nexon just uploaded massive predictive analytics data on the blob container from their legacy admin tool. When the batch task ends, events could be triggered by Azure Functions for update status and legacy monitoring tools. Without any infrastructure building, polling process or trigger daemon process, they only need to write code with their preferred development language.
|
||||
|
||||
The predicted analytics are expected to be settled on various industries in the near future. Building real-time prediction service with Azure App Service and batch predictive analytics task scheduler process with Azure Functions app will become comprehensive solutions and the best practice for predictive analytics solution scenarios.
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче