This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .
Designing modern applications that may include legacy components introduces unique challenges to the world of cloud architecture. IoT-related architecture can be different than other architecture.
The first thing to point out in this landscape is that there are two distinct scenarios: brownfield and greenfield. Anytime you’re building something new, such as a factory, building or car, and you want to include Internet connected devices or components, you are working within a greenfield scenario. Anytime you want to connect existing devices and assets that may have been around for years to decades, this typically is a brownfield scenario. The assets or devices were not designed to be network or internet connected by default. Thus, a retrofit methodology is required. In many cases, brownfield, while more difficult to work with and manage from an IoT perspective often are the reality for many organizations. Coupled with regulatory or security requirements on whether those devices should talk to the network or internet, there are several challenges that IoT solutions, specifically edge solutions, need to take into consideration.
In many cases, while it’s easier to start from scratch in a greenfield scenario, brownfield scenarios need to consider devices that may have been around forty years and have some notion of automation and connectivity but lack cloud-based concepts. Plus, many of these devices or manufacturing elements may use traditional telemetry and networked control infrastructure (such as SCADA and MES systems) that typically require effort to fit into standard cloud data models. There also might be preliminary stages of analytics or actions related to certain devices that need to be done at the edge. Thus, different levels and frequencies of message traffic will need to be incorporated into the system architecture, which may not have been as much an issue for greenfield solutions. But the key point is that IoT brings in unmalleable objects into the design and thus architecture considerations must adapt around these patterns.
Plus, architects need to maintain the context of the messages that devices send to the cloud.
Manufacturing plants send a finite number of messages
While manufacturing specifically is very complex and there's a lot of existing legacy that needs to be incorporated, the number of messages that a modern internet service, especially if you think about a consumer-based service, has to deal with is far more complex than what a manufacturing plant has to deal with. We have customers with very large manufacturing plants but ultimately they still have a finite number of message types that are fairly stable actually because the system produces what the system produces. Whereas an internet site can go into cycles dealing with large spikes of traffic that are much harder than dealing with the stable constants of manufacturing. However, for manufacturing scenarios, the volume of interconnected machines represent a high degree of demanding message generation, concurrency and transmission. Throughput and concurrency are essential to capture the volume of messages in this scenario both at the edge and the cloud.
The relevance of data determines how and where it’s processed, transmitted, and stored
A lot of data gets produced that has very short temporal value and it's not necessarily relevant for development of AI models, which is what a lot of people are trying to do for predictive maintenance. In some scenarios you prevent issues in the factory or in the plant or with a specific device that you're managing and it only has temporal value. But the value at that point in time is super important. Again, if you go back to a chemical process the temperature of the specific process element is really important. If it varies too much it might ruin the entire batch of processing, therefore you need to be able to react very fast in real time and therefore local processing is important. Whereas other elements you can run for longer processing because they're not necessarily relevant for this specific process but they are still relevant.
Some data is important for real-time reactions, whereas other data is more important for longer term processing and that means you need to figure out a way to get it into the cloud. If it's only needed for short term twitch reaction, then make sure that the process goes the right way you might not need to put it into the cloud. You might still want to archive it in the cloud but you might not want to push it in real time. Those are some of the decisions that make IoT architectures more complicated. Also, the variety of devices where factories have 20, 30, 40 years old equipment that wasn’t built for digital updates. How do you deal with that? Then you have a variety of operating systems and the hierarchy that are associated with it. If customers are using old technology then that’s obviously much more complicated than updating a modern Windows IoT or linux-based distribution.
IoT-specific design patterns
Are there different models that one uses when it comes to architecture for IoT than one
does for others? We talked about design patterns in the past. Are there design patterns that are more or less suited for IoT sort of situations, or were there any developed specifically for IoT?
The reuse part is. Certainly things like messaging, queuing, publish, and subscribe are very common inside IoT. We haven't talked about actors. Actor programming is very common and very popular in IoT because an actor can represent a device really well because it composes functionality and state in a single document. A lot of people choose to use the actor pattern to represent the IoT devices.
Data fusion is new in IoT because you have to fuse together a number of data sources to get to the ultimate data that you want to analyze. You can't simply say this is the pure data. You have to say it's only relevant if it's in this plant, in this production line, for example, so you need to contextualize the data. You might need to bring additional data sets like weather. Then what's very often the case is that the data set (out of the factory or plant or even the device) is incomplete so you need to bring other data sets in. That's what's called data fusion. That's a very common pattern in IoT.
There are other applications where that is also very common in AI development but in general it’s predominant in IoT. For example, there are others that are very specific and outside of specific standards, like opcwa in manufacturing and hl7 in healthcare. There's a whole bunch of these that are very specific but those are more of the same in terms of messaging, queueing, and pub sub capabilities.
Three main criteria that an IoT architecture has to satisfy
There are three coarse-grained buckets of an IoT architecture that must be satisfied. It has to connect in terms of being able to allow routes for devices to send telemetry to the edge and then extensively to the cloud. It has to collect that information, persist it somewhere where it can be looked at, and then analyzed in the future and maybe potentially used to train AI models but certainly answer questions. And then analyze, which is bringing together that data fusion component and bringing together all of the different components that are required.
How can we actually make sure that the solution does what it needs to do? The important factor here amongst those three coarse-grain solution components is that we do scale ingestion, we do the fusion of the data where we take time series telemetry, and we merge it with things like digital twin platforms for the context. And then we present that with metadata to the cloud so that people can analyze it through tools like Power BI or Tableau or those different types of analytical tools.
If you’d like to learn more and specific use cases, please watch the video below, or read the WAF for IoT post or register for the IoT Architecture Summit. Or just make a comment below!
To learn more about Azure’s Well-Architected initiative, go to our public page for an overview.