Delete _posts/2017-01-03-ZPMC.md
This commit is contained in:
Matt Zanner
GitHub
Родитель
ad30e1396a
Коммит
7ff0aaeb95
|
|
@ -1,505 +0,0 @@
|
|||
---
|
||||
layout: post
|
||||
title: "Azure IoT launches evolution of ZPMC port-machinery business"
|
||||
author: "Warren Zhou"
|
||||
#author-image: "{{ site.baseurl }}/images/authors/photo.jpg"
|
||||
date: 2017-06-26
|
||||
categories: [IoT]
|
||||
color: "blue"
|
||||
image: "images/ZPMC/feat_QC.png"
|
||||
excerpt: Microsoft and ZPMC teamed up to bring the ZPMC port-machinery business to the next level with Azure IoT services.
|
||||
verticals: [Discrete Manufacturing]
|
||||
language: [English]
|
||||
geolocation: [Asia]
|
||||
permalink: /iot/2017/02/15/ZPMC.html
|
||||
sticky: false
|
||||
---
|
||||
|
||||
*Updated with additional resources*
|
||||
|
||||
Shanghai-based ZPMC manufactures heavy-duty equipment that enables modern shipping. That equipment is a major source of real-time data, which needs to be retrieved, monitored, and analyzed. Microsoft Azure IoT Suite is helping ZPMC transform its business from traditional manufacturing to comprehensive service provider.
|
||||
|
||||
## Customer profile ##
|
||||
|
||||
Shanghai Zhenhua Heavy Industry Co., Ltd. ([ZPMC](http://www.zpmc.com/index.html)) is a leading heavy-duty equipment manufacturer, with A and B shares listed on the Shanghai Stock Exchange. ZPMC product lines include cranes, ship loader/unloaders, automated container systems, and even offshore platforms. ZPMC also owns a fleet of 26 transportation ships, with capacities from 60,000 [DWT](https://en.wikipedia.org/wiki/Deadweight_tonnage) to 100,000 DWT, delivering huge products or providing special transportation service all over the world.
|
||||
|
||||
*ZPMC quay cranes*
|
||||
|
||||
|
||||
|
||||
## Pain points ##
|
||||
|
||||
The current ZPMC port-machinery solution is on-premises and discrete. The programmable logic controllers (PLC) on machines such as quay cranes (QC), rail-mounted container gantry cranes (RMG), and automated guided vehicles (AGV) send thousands of signals every second to field [OPC](https://opcfoundation.org/) servers. The common way for ZPMC to monitor and retrieve device data is to log in to those OPC servers remotely. ZPMC needs an easier and more elegant way to collect the device data and get alarm notifications.
|
||||
|
||||
ZPMC also needs a business transformation to cope with the economy changes and global challenges, changing its role from a traditional equipment manufacturer to a comprehensive service provider in port planning, investing, construction, and operation.
|
||||
|
||||
To meet these needs, ZPMC requires an intelligent platform that can acquire real-time data from the machines that they are servicing all over the world and display that data in dashboards and interactive reports—plus generate global insights and enable predictive maintenance. ZPMC is also developing its next-generation automated port solution, which relies heavily on such a platform.
|
||||
|
||||
## Solution ##
|
||||
|
||||
The Microsoft developer engagement team worked with ZPMC on a machine-monitoring solution based on Azure services:
|
||||
|
||||
- Defining two types of data messages to IoT Hub: Alarm Data and Device Status Daily Data. The message structure is compatible with all their legacy port machines as well as future automated port equipment.
|
||||
|
||||
- Designing the solution architecture. The real-time data from port machines (QCs, RMGs, or AGVs) now can be collected by field OPC servers, subscribed to by unified OPC applications (agents), ingested by Azure IoT Hub, streamed to Stream Analytics, stored in Azure SQL Database and Blob storage, and monitored live in Power BI.
|
||||
|
||||
- Implementing the ZPMC IoT solution and integrating with QC devices in operation at Xiamen Port.
|
||||
|
||||
Key Microsoft technologies used in this project:
|
||||
|
||||
- OPC UA agent
|
||||
- Azure IoT Hub
|
||||
- Azure Stream Analytics
|
||||
- Azure SQL Database
|
||||
- Azure Blob storage
|
||||
- Power BI
|
||||
|
||||
## Architecture ##
|
||||
|
||||
This is the solution architecture of the ZPMC machine-monitoring solution.
|
||||
|
||||
|
||||
|
||||
|
||||
## Devices used and code artifacts ##
|
||||
|
||||
**Devices**
|
||||
|
||||
Because ZPMC manufactures dozens of types of port machines, they use all the major brands of PLCs in their machines to send status info and alarm signals to field OPC servers at port side. To avoid changes to the hardware and legacy system, neither the machines nor the OPC servers communicate with Azure IoT Hub. A new application would perform this job.
|
||||
|
||||
**Machine-data ingestion**
|
||||
|
||||
A stand-alone OPC agent (an application on Windows) was designed and implemented to subscribe to machine data with designated frequencies from multiple OPC servers. This application can run on Windows-based PCs either at port side (with Internet connections) or at the ZPMC remote monitoring center (with connection to the field OPC servers). The functions in the [Microsoft Azure IoT device SDKs](http://azure.github.io/azure-iot-sdks/) were integrated into this OPC agent.
|
||||
|
||||
The following code sends machine-data messages to IoT Hub.
|
||||
|
||||
```csharp
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Threading;
|
||||
using System.Text;
|
||||
using AzureHelper;
|
||||
using Opc.Da;
|
||||
using OpcNetBrowse;
|
||||
|
||||
namespace ziOPCClient
|
||||
{
|
||||
/// <summary>
|
||||
/// 消息分组，每个订阅对应一个消息分组
|
||||
/// </summary>
|
||||
class MessageGroup
|
||||
{
|
||||
public string GroupName;
|
||||
public bool IsAlarm = false;
|
||||
public string DeviceID;
|
||||
public int UploadRate;
|
||||
public Timer SendTimer;
|
||||
public List<AzureHelper01.OPCITEM> Messages = new List<AzureHelper01.OPCITEM>();
|
||||
public Mutex mutex = new Mutex();
|
||||
}
|
||||
|
||||
class AzureUploader
|
||||
{
|
||||
Dictionary<int, MessageGroup> m_groups;
|
||||
//AzureHelper01 m_azureHelper = new AzureHelper01();
|
||||
public Dictionary<string, AzureHelper01> m_azureHelper = new Dictionary<string, AzureHelper01>();
|
||||
public AzureUploader()
|
||||
{
|
||||
m_groups = new Dictionary<int, MessageGroup>();
|
||||
}
|
||||
public void AddOrUpdateGroup(int clientHandle, OpcGroup grp)
|
||||
{
|
||||
MessageGroup msg;
|
||||
if (m_groups.TryGetValue(clientHandle, out msg))
|
||||
{
|
||||
Logger.LogMessage("clientHandle exist:"+clientHandle);
|
||||
msg.GroupName = grp.GroupName;
|
||||
msg.IsAlarm = grp.IsAlarm;
|
||||
msg.DeviceID = grp.DeviceID;
|
||||
if(grp.UploadRate != msg.UploadRate)
|
||||
msg.SendTimer.Change(grp.UploadRate, grp.UploadRate);
|
||||
return;
|
||||
}
|
||||
|
||||
msg = new MessageGroup();
|
||||
msg.GroupName = grp.GroupName;
|
||||
msg.IsAlarm = grp.IsAlarm;
|
||||
msg.DeviceID = grp.DeviceID;
|
||||
msg.UploadRate = grp.UploadRate;
|
||||
msg.SendTimer = new Timer(new TimerCallback(OnSendTimer), msg, msg.UploadRate, msg.UploadRate);
|
||||
|
||||
m_groups[clientHandle] = msg;
|
||||
}
|
||||
public void RemoveGroup(int clientHandle)
|
||||
{
|
||||
MessageGroup group;
|
||||
if (!m_groups.TryGetValue(clientHandle, out group))
|
||||
{
|
||||
return;
|
||||
}
|
||||
group.SendTimer.Dispose();
|
||||
group.Messages.Clear();
|
||||
group.mutex.Dispose();
|
||||
m_groups.Remove(clientHandle);
|
||||
}
|
||||
private string ValueToString(object value)
|
||||
{
|
||||
if (value == null)
|
||||
return "";
|
||||
if (value.GetType().IsArray)
|
||||
{
|
||||
StringBuilder str = new StringBuilder();
|
||||
foreach (object item in value as Array)
|
||||
str.Append(item.ToString()).Append(",");
|
||||
return str.Remove(str.Length - 1, 1).ToString();
|
||||
}
|
||||
return value.ToString();
|
||||
}
|
||||
|
||||
private int GetDataType(object val)
|
||||
{
|
||||
int nDataType = 0;
|
||||
if (val == null)
|
||||
return nDataType;
|
||||
if (val.GetType().IsArray)
|
||||
{
|
||||
nDataType = (int)System.Type.GetTypeCode(val.GetType().GetElementType()) | Define.DATATYPE_ARRAY;
|
||||
}
|
||||
else
|
||||
{
|
||||
nDataType = (int)System.Type.GetTypeCode(val.GetType());
|
||||
}
|
||||
return nDataType;
|
||||
}
|
||||
|
||||
public void AddMessage(int clientHandle, ItemValueResult[] values)
|
||||
{
|
||||
MessageGroup group;
|
||||
if (!m_groups.TryGetValue(clientHandle, out group))
|
||||
{
|
||||
return;
|
||||
}
|
||||
group.mutex.WaitOne();
|
||||
foreach (ItemValueResult itemResult in values)
|
||||
{
|
||||
AzureHelper01.OPCITEM item = new AzureHelper01.OPCITEM();
|
||||
item.ItemName = itemResult.ItemName;
|
||||
item.DataType = GetDataType(itemResult.Value);
|
||||
if (item.DataType == (int)TypeCode.String
|
||||
|| item.DataType == (int)TypeCode.DateTime
|
||||
|| (item.DataType & Define.DATATYPE_ARRAY) == Define.DATATYPE_ARRAY)
|
||||
{
|
||||
item.ValueString = ValueToString(itemResult.Value);
|
||||
item.ValueFloat = -1;
|
||||
item.ValueInt = -1;
|
||||
}
|
||||
else if (item.DataType == (int)TypeCode.Single
|
||||
|| item.DataType == (int)TypeCode.Double
|
||||
|| item.DataType == (int)TypeCode.Decimal)
|
||||
{
|
||||
item.ValueFloat = System.Convert.ToSingle(itemResult.Value);
|
||||
item.ValueString = null;
|
||||
item.ValueInt = -1;
|
||||
}
|
||||
else
|
||||
{
|
||||
item.ValueInt = System.Convert.ToInt32(itemResult.Value);
|
||||
item.ValueString = null;
|
||||
item.ValueFloat = -1;
|
||||
}
|
||||
item.TimeStamp = itemResult.Timestamp;
|
||||
item.Quality = itemResult.QualitySpecified;
|
||||
group.Messages.Add(item);
|
||||
}
|
||||
group.mutex.ReleaseMutex();
|
||||
}
|
||||
public void Stop()
|
||||
{
|
||||
foreach(KeyValuePair<int, MessageGroup> pair in m_groups)
|
||||
{
|
||||
pair.Value.SendTimer.Dispose();
|
||||
pair.Value.Messages.Clear();
|
||||
pair.Value.mutex.Dispose();
|
||||
}
|
||||
m_groups.Clear();
|
||||
}
|
||||
|
||||
public void OnSendTimer(object state)
|
||||
{
|
||||
MessageGroup group = state as MessageGroup;
|
||||
if (group == null || group.Messages.Count == 0)
|
||||
return;
|
||||
try
|
||||
{
|
||||
group.mutex.WaitOne();
|
||||
while (group.Messages.Count > 0)
|
||||
{
|
||||
AzureHelper01.OPCMessage msg = new AzureHelper01.OPCMessage();
|
||||
msg.DeviceID = group.DeviceID;
|
||||
msg.IsAlarm = group.IsAlarm;
|
||||
msg.GroupName = group.GroupName;
|
||||
|
||||
if (group.Messages.Count < Define.MAX_MSG_SIZE)
|
||||
{
|
||||
msg.Items = group.Messages.ToArray();
|
||||
group.Messages.Clear();
|
||||
}
|
||||
else
|
||||
{
|
||||
msg.Items = group.Messages.GetRange(0, Define.MAX_MSG_SIZE).ToArray();
|
||||
group.Messages.RemoveRange(0, Define.MAX_MSG_SIZE);
|
||||
}
|
||||
m_azureHelper[group.DeviceID].sendDeviceTelemetryDataToIOT(msg);
|
||||
}
|
||||
group.mutex.ReleaseMutex();
|
||||
}
|
||||
catch (Exception e)
|
||||
{
|
||||
System.Diagnostics.Trace.Write(e.Message);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The following code is the helper class that sends and receives messages between the code and IoT Hub.
|
||||
|
||||
```csharp
|
||||
using Microsoft.Azure.Devices.Client;
|
||||
using Newtonsoft.Json;
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Diagnostics;
|
||||
using System.Linq;
|
||||
using System.Runtime.Serialization;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace AzureHelper
|
||||
{
|
||||
public class AzureHelper01
|
||||
{
|
||||
|
||||
[DataContract]
|
||||
public class OPCITEM
|
||||
{
|
||||
[DataMember]
|
||||
public string ItemName;
|
||||
[DataMember]
|
||||
public int DataType;
|
||||
[DataMember]
|
||||
public string ValueString;
|
||||
[DataMember]
|
||||
public float ValueFloat=0;
|
||||
[DataMember]
|
||||
public int ValueInt=0;
|
||||
[DataMember]
|
||||
public bool Quality;
|
||||
[DataMember]
|
||||
public DateTime TimeStamp;
|
||||
}
|
||||
|
||||
[DataContract]
|
||||
public class OPCMessage
|
||||
{
|
||||
[DataMember]
|
||||
public string DeviceID;
|
||||
[DataMember]
|
||||
public bool IsAlarm;
|
||||
[DataMember]
|
||||
public string GroupName;
|
||||
[DataMember]
|
||||
public OPCITEM [] Items;
|
||||
}
|
||||
|
||||
private DeviceClient deviceClient;
|
||||
|
||||
Task ReceivingTask;
|
||||
public AzureHelper01()
|
||||
{
|
||||
|
||||
}
|
||||
private bool checkConfig()
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
private byte[] Serialize(object obj)
|
||||
{
|
||||
string json = JsonConvert.SerializeObject(obj);
|
||||
return Encoding.UTF8.GetBytes(json);
|
||||
|
||||
}
|
||||
|
||||
private dynamic DeSerialize(byte[] data)
|
||||
{
|
||||
string text = Encoding.UTF8.GetString(data);
|
||||
return JsonConvert.DeserializeObject(text);
|
||||
}
|
||||
|
||||
public async void sendDeviceTelemetryDataToIOT(OPCMessage opcMsg)
|
||||
{
|
||||
try
|
||||
{
|
||||
var msg = new Message(Serialize(opcMsg));
|
||||
if (deviceClient != null)
|
||||
{
|
||||
await deviceClient.SendEventAsync(msg);
|
||||
}
|
||||
}
|
||||
catch (System.Exception e)
|
||||
{
|
||||
Debug.Write("Exception while sending device telemetry data :\n" + e.Message.ToString());
|
||||
}
|
||||
Debug.Write("Sent telemetry data to IoT Suite" );
|
||||
}
|
||||
|
||||
public async void connectToIoTSuite(string connectionString)
|
||||
{
|
||||
|
||||
try
|
||||
{
|
||||
deviceClient = DeviceClient.CreateFromConnectionString(connectionString, TransportType.Http1);
|
||||
await deviceClient.OpenAsync();
|
||||
|
||||
ReceivingTask = Task.Run(ReceiveDataFromAzure);
|
||||
}
|
||||
catch
|
||||
{
|
||||
Debug.Write("Error while trying to connect to IoT Hub");
|
||||
deviceClient = null;
|
||||
}
|
||||
}
|
||||
|
||||
public async void disconnectFromIoTSuite()
|
||||
{
|
||||
if (deviceClient != null)
|
||||
{
|
||||
try
|
||||
{
|
||||
await deviceClient.CloseAsync();
|
||||
deviceClient = null;
|
||||
}
|
||||
catch
|
||||
{
|
||||
Debug.Write("Error while trying close the IoT Hub connection");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
The following code is the message data structure sent to IoT Hub for processing by Stream Analytics. Although the size of the message sent to IoT Hub varies depending on the load status of the machine, and the sending frequency can be set by the OPC agent, we can still reasonably estimate that monitoring a medium-load quay crane requires less than 8 KB of data every second.
|
||||
|
||||
```csharp
|
||||
[DataContract]
|
||||
public class OPCITEM
|
||||
{
|
||||
[DataMember]
|
||||
public string ItemName;
|
||||
[DataMember]
|
||||
public int DataType;
|
||||
[DataMember]
|
||||
public string ValueString;
|
||||
[DataMember]
|
||||
public float ValueFloat=0;
|
||||
[DataMember]
|
||||
public int ValueInt=0;
|
||||
[DataMember]
|
||||
public bool Quality;
|
||||
[DataMember]
|
||||
public DateTime TimeStamp;
|
||||
}
|
||||
[DataContract]
|
||||
public class OPCMessage
|
||||
{
|
||||
[DataMember]
|
||||
public string DeviceID;
|
||||
[DataMember]
|
||||
public bool IsAlarm;
|
||||
[DataMember]
|
||||
public string GroupName;
|
||||
[DataMember]
|
||||
public OPCITEM [] Items;
|
||||
}
|
||||
```
|
||||
|
||||
**Security consideration**
|
||||
|
||||
Although the data transmission is on proprietary links from the machines to field OPC servers and then to the OPC agent, the data sent from the OPC agent to IoT Hub must travel over the Internet. In this project, we chose the [AMQP protocol](https://www.amqp.org/) to enhance the security of the device-to-cloud communication, which is natively supported by IoT Hub.
|
||||
|
||||
For the security of data storage in the cloud, some easy-to-configure security practices such as transparent data encryption for Azure SQL Database were adopted to protect the digested alarms and statistics data.
|
||||
|
||||
**Stream Analytics**
|
||||
|
||||
Azure Stream Analytics parses ingested machine status and alarms from IoT Hub and sends the data to Blob storage (for data archive) and SQL Database (for digested and latest reports), as well as to the Power BI visualization.
|
||||
|
||||
*Stream Analytics input configuration*
|
||||
|
||||
|
||||
|
||||
*Stream Analytics output configuration*
|
||||
|
||||
|
||||
|
||||
**Data visualization in Power BI**
|
||||
|
||||
Power BI renders the real-time machine-status data from IoT Hub and the processed alarms from SQL Database and presents to ZPMC interactive reports on the ports and machines.
|
||||
|
||||
*Power BI report: ports, locations, and machine numbers*
|
||||
|
||||
|
||||
|
||||
*Power BI report: machine status and 24-hour alarms with port and category slicers*
|
||||
|
||||
|
||||
|
||||
## Opportunities going forward ##
|
||||
|
||||
Based on the current solution prototype, ZPMC would integrate more ports and machines data and build up a global remote monitoring center. Besides that, the capability of maximizing the value of their device data and providing predictive maintenance service to their customer is also on very top of ZPMC's wish list. Such a proactive and predictive service other than the traditional 'report-response' way would surely impress ZPMC's customers and help ZPMC keep significant advantages against their competitors.
|
||||
|
||||
## The team ###
|
||||
|
||||
ZPMC Electric Division:
|
||||
|
||||
- Ting Zhu, Software Technical Manager
|
||||
- Jun Zhao, Software Engineer
|
||||
- Xiaoting Wang, Software Engineer
|
||||
|
||||
Microsoft DX China:
|
||||
|
||||
- Warren Zhou, Sr. Technical Evangelist
|
||||
- Lit Li, Prin. Technical Evangelist
|
||||
- Michael Li, Technical Evangelist
|
||||
|
||||
Special thanks to Michael SH Chi, Software Development Engineer in Taiwan, for his active participation in the architecture discussion and hackfests.
|
||||
|
||||
*The ZPMC and Microsoft DX teams at the hackfest*
|
||||
|
||||
|
||||
|
||||
*Draft of the architecture*
|
||||
|
||||
|
||||
|
||||
*The teams investigating SA bugs*
|
||||
|
||||
|
||||
|
||||
## Conclusion ##
|
||||
|
||||
>"IoT Hub is one of the key Azure services we used in the project. After we evaluated and used the IoT Hub service, we believe that it could remarkably ease our burdens to build a scalable, efficient, and easy-to-maintain data ingestion infrastructure. During the joint hackfest, the experts from Microsoft did a great job and helped us solve all the major issues from project architecture to data ingestion and to data visualization. Surely, this project would make a solid foundation to build up our predictive maintenance capability." —ZPMC
|
||||
|
||||
The work in this project will help ZPMC not only get returns immediately but also build a solid foundation for their core competence in the long run. Reports from multiple machines and even ports can be easily generated, which saves operation workload and cost. And being able to collect real-time data from any machine in any port unlocks the next stage for ZPMC: building their predictive-maintenance capability, which can lead the port-machinery industry to the next level.
|
||||
|
||||
Leveraging the intelligent cloud capabilities of Azure services, ZPMC is now on the fast track of digital transformation, which will help them maintain their global leadership in port-machinery servicing.
|
||||
|
||||
## Additional resources
|
||||
|
||||
- Explore [Azure IoT Hub documentation](https://docs.microsoft.com/en-us/azure/iot-hub/)
|
||||
- Find IoT devices and starter kits: [Azure IoT device catalog](https://catalog.azureiotsuite.com/kits)
|
||||
- Try any Azure services for free: [Create your free Azure account today](https://azure.microsoft.com/en-us/free/)
|
||||
- Check out a curated collection of IoT learning resources: [Microsoft Technical Community Content](https://github.com/Microsoft/TechnicalCommunityContent/tree/master/IoT) on GitHub
|
||||
- Read more IoT-focused [technical case studies](https://microsoft.github.io/techcasestudies/#technology=IoT&sortBy=featured) (like this one)
|
||||
Загрузка…
Ссылка в новой задаче