Our client
The company is a startup founded by veterinary experts who wanted to create smart solutions for animal healthcare.
They aimed to create wearables and complete products dedicated to professional veterinarian caregivers like animal clinics, hospitals, etc.
Challange
We were tasked with taking care of the project as an interim product owner. We needed to create a viable product that could be tested. It involved both software and hardware aspects.
We were initially tasked with creating a machine algorithm for the device. After conducting discovery workshops, our team proposed a new roadmap for the product.
Solution
The team of experts proposed a deep-learning approach for the initial classification and assessment of the severity of the illness. The data could be later extracted and used to monitor the disease further.
After identifying the necessary steps to clean the provided data, we employed deep learning models and a deep convolutional neural network based on raw data for pattern recognition.
The startup was founded by veterinary experts with a plan to support professionals and veterinary clinics with technological solutions. They planned to develop an end-to-end solution that would help diagnose osteoarthritis disease among dogs.
The device’s design had to be compact and light enough to attach to the dog’s collar easily. It would then record the movements while walking or running to pinpoint indicators that could designate potential symptoms of OA, alerting the owner to look for further diagnosis. With the vast domain knowledge, our client needed technology experts to complete the product’s technical part.
The first significant challenge involved a tight budget and time. The company has already tried cooperating with other tech companies. However, the results could have been better. Thus, a lot of resources have been wasted. They needed a partner to help progress with the product to be viable enough for testing if it could work and be accurate enough to invest in developing the final product.
Before developing the algorithm, our team (also comprising PhD-level specialists) needed to evaluate the available data collected. Upon examining it, our team noticed that the data wasn’t correctly annotated. There was also an issue with the logistic regression model and overall technical trouble with computer-aided detection and data collection from the gyroscope.
We’ve decided to employ deep learning models and deep convolutional neural networks for imaging data to recognise patterns based on collected raw data. We wanted to prove the universality of solutions of the deep learning model for different breeds. At first, we relied on the previously collected training data provided to us by data scientists.
Our solution involved the development of the right processing artificial intelligence algorithms and AI models of the data from accelerometers and gyroscopes, then creating classification algorithms that operate on the collected data sets and adjusting the software/hardware interfacing to obtain stable results.
Machine learning and artificial intelligence were at the heart of the project. Our task was to conduct data analysis based on the provided data sets and input data, with the help of computing resources, artificial intelligence and machine learning model, with measurements of dog movements and to distinguish between dogs with and without OA based on the collected sensor’s raw data.
Based on the training data provided by data scientists, we started by monitoring dogs’ activity over a long period due to the importance of high precision of deep machine learning and digital imaging. After analysing different data points and input and output variables, we aimed to introduce significant improvements to the solutions regarding logistic regression models, machine learning and AI models. We sought discriminative features and data points crucial for determining dogs’ illnesses. Based on the results, using analytical methods, we were able to determine whether the dog had OA.
Our team proposed a deep neural network machine learning algorithm to increase the level of classified OA severity and the diagnostic accuracy of the device and AI models. The results of our efforts allowed us to sort through all the data and extract the deep features. We implemented artificial intelligence solutions and deep neural networks to distinguish between dogs with OA symptoms. We proposed a high-quality deep-learning model and other deep-learning technologies to generate results that could work with more data in the future.
Results
Our experts and the client were happy with the results achieved during our cooperation. We were able to extract all the necessary data and implement further measures. The speed of the delivery and the quality of provided solutions left the product in a state that allowed for further testing and development of its final stages. Our team built a solid and reliable foundation for the company to proceed with the development upon receiving funds.
Important figures
About the client
Problem overview
Swiftly, a Stockholm-based startup, grappled with two significant challenges within their job portal. Firstly, accurate categorization of job listings posed difficulties, leading to suboptimal user experiences and ineffective job matching. Secondly, the manual job application process was time-consuming and resource-intensive, restricting scalability.
Proposed solution
Our approach comprised two pivotal components:
Collaborative discussions between Swiftly’s leadership and DAC.digital’s technical team laid the groundwork for a productive partnership, aligning expectations and goals.
An 8-person team, comprising ML Engineers, Embedded Systems Engineers, Data Scientists, and Fullstack Developers, came together to tackle the project.
Daily stand-up meetings and ongoing communication facilitated iterative development and enhancements.
The project concluded with the creation of an advanced SOTA solution that effectively addressed Swiftly’s challenges. This solution improved job listing categorization precision and streamlined the application process. The SOTA also offered a proof-of-concept for refining job listing keywords and automating application field population.
Results
Swiftly’s collaboration with DAC.digital resulted in the successful resolution of their job portal challenges through the implementation of innovative automation solutions. The web scraping tool and the SOTA presentation highlighted the potential of technology to enhance processes, elevate user experiences, and pave the way for future enhancements.
Key numbers
Our Client
Challenge
The client noticed that the dairy industry was heavily reliant on traditional practices, leading to inefficiencies. Recognizing the opportunity for improvement, the client aimed to create a digital system that could streamline milk delivery management and procurement processes.
Solution
DAC.digital’s team collaborated closely with the client to create a comprehensive end-to-end product. We decided together on the features and improved the concept collaboratively.
The team comprised a Product Owner, 3 backend developers, 2 frontend developers, a UI/UX designer, and 1 tester. As the project required, the team size was adjusted accordingly, growing when new functionalities were needed and scaling down during lower business demand.
To accommodate the complex and evolving nature of the system, we opted to work in a Time and Materials model. This approach provided the necessary flexibility and responsiveness to adapt to the project’s changing needs over time.
As we continued to enhance the system, the client recognized the need to optimize the milk delivery process further. To achieve this, our team introduced automatic route planning algorithms, which revolutionized how routes were planned for the drivers. Instead of relying on manual decision-making, the system could generate the most efficient routes based on various factors like delivery locations, vehicle capacity, and traffic conditions. This saved time, reduced fuel consumption, and improved overall operational efficiency.
Additionally, we sought to digitize and simplify the process of documenting milk quantities at each farm. To accomplish this, we developed electronic devices called MuuBox. These devices were installed in milk tankers and are responsible for uploading data in real time to the MuuMap system.
Previously, the milk collection process involved a lot of manual work for the drivers. They either had to use handwritten protocols to record the quantity of milk collected at each farm or print bulk receipts from their route, which had to be manually entered into the computer. This manual data entry was a time-consuming and labor-intensive task. For instance, entering 3000 positions manually required significant effort.
However, with the implementation of our solution, this manual process was digitized and streamlined. The collected pick ups data is now automatically aggregated in our system and allows integration with others. This automation significantly reduced the need for manual data entry, resulting in fewer errors than the previous approach.
By automating the data entry process, MuuBox significantly reduced the administrative burden on both drivers and the milk collection department. Instead of being the source of mistakes, the department can now focus on correcting errors.
For those MuuMap clients who preferred not to use the MuuBox devices, we introduced a manual entry functionality directly on their tablets. With this option, they could input the milk quantities manually. The advantage of this approach was that it eliminated the need for paperwork while ensuring accurate data recording. Just like with the MuuBox integration, the collected data from the manual entries were uploaded in real-time to the MuuMap system, streamlining the process and ensuring seamless data management.
Thanks to the application, the client became a pioneer in the market by offering a tool specifically designed for the traditional dairy industry. The unique and efficient solution attracted the first customers, who loved it and spread positive reviews. As a result, the client’s reputation grew, and more and more people started using the application. Eventually, it captured a significant portion of the Polish dairy market, securing a dominant position with a 30% market share.
How we helped patients communicate with their caregivers in an emergency
Who did we work with?
Our client is a representative of the Nursing and residential care industry, a part of the MedTech sector.
They work on different solutions that may help the caregivers keep an eye on their patients, in case an emergency happens outside their working hours.
Their most important goal is to ensure that their patients can communicate them effortlessly in any situation that would require immediate assistance.
What was the challenge?
Due to a long prototyping phase, the company had a working product. However, the length of the prototyping deemed some of the components had gone out of date and required replacements or a rework of the infrastructure. The outbreak of the Covid-19 pandemic made some of the parts hard to access, especially the semiconductors, causing turmoil in the market.
With limited documentation and resources, our team needed to work out a solution that would compromise on some resources and accommodate the end users’ specific needs.
How did we want to tackle it?
With the limited documentation and outdated documents, our team had to work out a viable solution to fit the budget.
After detangling the code and failed architecture elements from the working devices, the experts proposed redesigning the product to work with the new components that needed replacement.
It required a combination of software and hardware to create a working hub to integrate the devices into the system.
Our partner, a MedTech company specialising in implementing innovative solutions for nursing homes and in-home patient care needed an easy-to-use device to allow patients to communicate with their caregivers in an emergency. It resulted from the fact that the caregivers in some countries work only during business weekdays, and patients living alone in their own homes needed an infallible means to communicate in urgent need.
The device needed to be very easy to use, as senior patients often aren’t tech-savvy enough to work with more complex technology. It had to accommodate the key functions and enclose them in several buttons and clear displays.
The product went through a long, six-year prototyping phase and at the end, some of the components were outdated and required a replacement or different infrastructure to work properly. They needed hardware and software experts to assemble the elements into a working IoT network hub.
Our client needed expertise and a competent team in both hardware and software to help implement the required functions and the device connectivity to the system. They met our experts, and the project started in 2021.
After meeting the client and familiarising ourselves with them, our team had many challenges to face so that they could outline an optimal solution.
Due to the long prototyping phase and the need to change some key components, our team deemed the existing architecture and software insufficient. They also had to deal with limited documentation regarding the devices and their infrastructure.
After navigating the available resources and consulting the elements with the lead developer and the CEO to lay out the problem, the team pointed out that the infrastructure needed a rework due to the outdated components and parts of the code.
Our team comprised a project coordinator, delivery manager, tech lead, BLE expert and an electronic engineer to ensure software and hardware implementation. They were supported by the company’s lead developer and CEO, who actively participated in the development and helped fill in the gaps whenever necessary.
To deliver a device that could be a part of the bigger IoT network, the team must manage and organise their work based on the available documentation and expertise of the company’s lead developer.
After untangling the issues with new components, existing architecture and code, they proposed redesigning the architecture and rewriting the code, as it was easier than trying to work out how to incorporate the existing code and infrastructure into new components.
They also implemented the necessary features into an easy “press-and-play” device that would pose no problem for end users. The BLE technology proved useful in helping manage battery life optimisation.
Assessing the state of the current software and architecture and how it can work with the device
Redesigning the product to work with new components
Designing new firmware to work with the existing devices
Code analysis to make it compatible with new components
Extending the battery working time from the initial 12h using one battery
Implementing energy management optimisation
Ensuring that the software and hardware comply with the EU MedTech norms
New optimisation and code base
BLE implementation
The cooperation ended in November 2022, and the client was satisfied with the results. We created a solid foundation for the product that is currently available on the market. Our continuous efforts to improve and enhance the product helped the company to reduce the market release time. Throughout the collaboration, the product has been continuously supported by our developers.
Customer
Data management
Software integration
IoT
The simple provision of chargers to tenants and billing of energy consumed in the administration system.
Optimal use of available power in the building.
Protection against network overload in an office building or parking lot.
Here we have used PostgreSQL algorithms (ltree) for representing labels of data stored in a hierarchical tree-like structure.
We have employed a PostGIS, a spatial database extender for PostgreSQL object-relational database. It supports geographic objects allowing location queries to be run in SQL.
We have deployed an open-source message broker, RabbitMQ, which can be deployed in distributed and federated configurations to meet high-scale, high-availability requirements.
The goal was to provide remote access to the status of the charger but also to allow users and end-users to control the charger, e.g., switch it on/off remotely. Both groups are using different apps to perform these activities. Our team accessed the chargers software backend and conducted the integration from that level.
DAC.digital software development team has been working together with the client’s team. The work has been aligned with the scrum methodology.
The system allows dividing the network into smaller operators, who will only have access to their devices. Charging Service Providers can check the status of the charging station. Thanks to the connection with the Enelink system, most of the maintenance activities will be performed remotely. Another feature was setting up a charging plan that will limit the station’s power at the right time if the Service provider chooses several Operators. All charging stations can be labeled. This makes it easier for Service Provider to manage the stations from a given label. Then, information about the stations’ availability can be easily shared with end-users in a few clicks.
Dynamic Tariff solution gives an attractive offer for each end-user, encourages them to charge in specific places, and introduces discount coupons and VIP programs. Entering tariffs helps Service Providers optimize earnings at charging stations.
Customer.
Mobility
Data Analytics
IoT
The Company was looking for improvements in the following areas:
MuuMap provides support for managing vehicle fleets and transport personnel. The system allows its users to edit maps, supplement them with new roads, and mark temporarily unavailable routes, as well as enter data about the locations of suppliers’ locations and their milk volume. The fleet management module allows the identification of vehicle capacity, the number of compartments, and the option of attaching trailers. Next, the information is used for optimizing and planning milk collection. MuuMap also offers a vehicle registries and facilitates periodic settlements of the costs of fleet maintenance.
The company reported saving of up to 20% kilometers and 30% total cost and conserved 50% planning time. The system makes it possible to rotate drivers on the routes and it has been lowered by up to 80% the induction training time. The platform allows employees for fast access to all real-time information about transport: fleet time, supplier and recipient characteristic and locations, current stage of realization of the routes, location on the fleet and drivers.
Our Client
Challenge
The company was looking for solutions to the following problems:
Solution
We have provided Eldro with a wide range of solutions clustered into the comprehensive and easy to operate platform, including:
Cooperation with Eldro involved creating the system from scratch – from the concept to the implementation of full functionality. Despite the production team, we also appointed a Product Owner, who oversees the process of creating requirements and specification. The project involved creating an interface design. Therefore a UX Specialist was involved. Due to the specificity of the market, the project did not end in a single product but is still subject to development and changes.
The possibility of appearing in new tenders in new areas marketers not yet available.
Reduction of the costs of handling the contract.
Inclusion of costs related to contractual penalties for failure to comply with SLA conditions
Our Client
Challenge
The Company has been dealing with the following issues:
Solution
The cooperation started with a detailed audit of the equipment was carried out, which had transporters carrying out the process of milk collection.
The process is fully automatic and does not require the participation of dairy workers, manually completed prints have been eliminated.
One employee from the Purchase department who was able to enter receipts manually was moved.
Drivers’ work has been improved.
Fraud has been limited.
Reporting time has been limited.
SIN-On is an onboarding toolchain for IoT Nodes (based on STM32WB55 microcontrollers) designed to be seamless and user-friendly (the user requires NO technical knowledge). It makes the deployment of sensor networks quick and easy.
The toolchain has a management layer in the cloud that is used to manage sensors’ configuration (e.g. sampling frequency). Hence, with some simple operations, the user can decide which sensor has to return its measures and at which frequency. This, in terms of time, is really efficient because the user does not have to “manually” add this information in the configuration files of each sensor and neither re-program the microcontroller to get the actual data.
It is estimated that by 2026, over 64 billion devices will be a part of the IoT ecosystem. As the number of IoT devices grows, the need for integrating the new devices into the IoT network poses several challenges. For instance, Derhamy et al. evaluated the existing IoT frameworks against criteria such as security, protocols, rapid application development support, hardware requirements, architectural approach, interoperability, industry support etc. They found that there is a need to develop a more advanced framework and tool that would help businesses to quickly and seamlessly integrate new IoT nodes/devices into their existing system or IoT Infrastructure. Although Derhamy et al. deduced this in 2015, not much progress was done in this domain as confirmed by Paniagua and Delsing in 2021.
The solutions which already existed prior to the development of SIN-On were limited by some or other barrier. Some of them were limited by the number of devices that they could onboard, whereas others were either limited by the type of devices they could onboard or some other dependencies such as manufacturing or distribution. For SIN-On the theoretical limit is known and ranges in a few hundreds of sensors per gateway, however the practical limit is a bit lower. There was also a lot of manual effort required from the user, or a specialist was required each time a new node had to be onboarded to the existing system. Compatibility issue was also quite common in most industries. This amounted to higher cost, and lag time, which was not so beneficial for the businesses.
Hence the main motivations for the development of SIN-On were as follows:
The credentials of a node (stored in the secure memory of the Hardware Security Module) are scanned through a phone’s NFC, which takes the user to the onboarding web-app. The onboarding app (after logging in) asks for the name of a node, the gateway to which it should be onboarded, and one of the preset configurations to onboard the node. After that, the data are sent to Cloud Management Interface, which passes the configuration to a particular gateway. The gateway starts to look for the unpaired nodes, and once the node is powered up and in range of the gateway, they pair together, the gateway sends the configuration to the node (through GAP GATT REST API), and the node starts providing the data. The data are passed to influxdb and visualized in real-time.
This onboarding toolchain makes deploying new sensor networks easily manageable, configurable, and secure. It is compatible with your own embedded or cloud solution, where the measurements can be processed on board or via a distributed application.
GAP GATT REST API is a tool developed by DAC.digital as a part of the onboarding toolchain developed in the use case 11 (UC11) of Arrowhead Tools: Configuration tool for autonomous provisioning of local clouds. The motivation behind the development of this tool was to incorporate the Bluetooth Low Energy (BLE) based data provisioning in the Service-oriented Architecture (SOA) ecosystem.
The tool is mainly designed to work on linux-based embedded devices, although it should work also on desktop computers with linux-based systems and bluetooth chips. It consists of two parts – Generic Access Profile (GAP), for the management of BLE connections and their parameters, and Generic Attribute Profile (GATT) for exchanging the data which are described in terms of their functionalities below.
Generic Access Profile: GAP REST API provides endpoints, among others, for: Passive and active scanning of the nodes, connecting to a node GAP is stateless, and supports a gateway operating as central and observer roles. The Hyper Text Transfer Protocol (HTTP) method GET is translated to Bluetooth’s READ, while PUT – to WRITE. Additionally to PUT and GET, the EventSource method is used to handle a stream of notifications and indications.
Generic Attribute Profile: GATT supports, among other, the following functionalities: – Service discovery (as well primary services and filtering by Universal Unique Identity (UUID)) – Characteristics discovery (by UUID as well) – Subscribing to notifications and indications – Reading and writing characteristics – Handling notifications and indication on the server side – Reading/writing descriptors of characteristics
New developments and recent trends in wireless sensor networking technologies have sparked the creation of inexpensive, low-power, multipurpose sensor nodes. Data processing and environment sensing are made possible by sensor nodes. Different surroundings may be monitored with the use of sensors that can detect volatile compounds, temperature, and humidity. They can communicate via networks with other sensor devices and share data with outside users.
The evolution of the Web as we know it has been influenced by the Representational State Transfer (REST) architectural style, which outlines a set of guidelines for the design of networked hypermedia systems. RESTful Web services are web services that adhere to the REST architectural style, and REST APIs are the programmatic interfaces for these services. The architectural decisions made by the Web to support the scalability and stability of networked, resource-oriented systems based on HTTP have greatly influenced the design concepts for REST APIs. The core principles are: Resource addressability, resource representations, uniform interface, statelessness, hypermedia as the engine state.
The use of GATT REST API and GAP REST API for Bluetooth low energy devices was introduced back in 2013. Since there they have been seen in several applications. GAP defines the general topology of the BLE network stack. GATT describes in detail how attributes (data) are transferred once devices have a dedicated connection.
In this context, one of the main motivations of this solution was that Legacy data exchange technologies are not supported by modern SOA/microservices architectures. Moreover, the deployment of complex sensor networks, their management, diagnosis, decommissioning, and evolution of the sensor network, required specialised knowledge and tools. Hence to overcome this limitation, an onboarding tool was developed, of which GAP GATT REST API was a part.
The tool is implemented according to Bluetooth Special Interest Group (Bluetooth SIG) specifications (GAP REST API V10r01 and GATT REST API V10r01) and is a natural proxy/translator between BLE communication interfaces and REST API (used by Eclipse Arrowhead).
Arrowhead Compatibility: To make the tool compatible with Arrowhead, a specially designed Attribute Table is used to differentiate measurements (with value characteristics) and metadata about the device. Both measurement service and metadata service are registered in Arrowhead Service Registry as soon as they are connected to the gateway (in UC11 there’s a tool responsible for automated connection with the BLE nodes). At the same time, the authorization and orchestration rules are created, and the gateway that requested the connection can consume the incoming data from the provider.
To make the tool compatible with Arrowhead, a specially designed Attribute Table is used to differentiate measurements (with value characteristics) and metadata about the device. Both measurement service and metadata service are registered in Arrowhead Service Registry as soon as they are connected to the gateway (in UC11 there’s a tool responsible for automated connection with the BLE nodes). At the same time, the authorization and orchestration rules are created, and the gateway that requested the connection can consume the incoming data from the provider.
D-Box with Shipping Node for Food Transportation Read More
BARTI with Shipping Node for Container Management Read more
The measurements are obtained using the onboard computer (the list of measurable variables is extracted from the setup description) and sent to the Ethereum blockchain (to oraclize the data). Oraclize is a service that allows smart contracts to access data from other blockchains and the Internet. The measured data and the hash of the blockchain transaction are communicated to the backend using IDSCP (Industrial Data Spaces Communication Protocol) via the IDS connector, where they were saved in MongoDB and visualised on the front end in real-time.
The Data Generation Application (DGA) replicates the operation of a milk truck measuring system. The data is created in JSON format (for the time being) and is provided over IDS to an external consumer, who saves it in the database.
DGA also transmits data to the Distributed Ledger Technology (DLT) connection, which stores it on a Distributed Ledger and provides the transaction hash.
The Arrowhead Framework is designed to act as a service discovery interface between the OBC application and the Digital Reference Platform. The sequence may look like this:
N-SAAW is a one-of-a-kind Deep Neural Network (DNN)-based system for monitoring farm animal health and wellbeing. This method was created specifically to examine the milk protein to fat ratio, encompassing extremes associated with malnutrition. This type of detection aids in lowering the danger of ketosis or acidosis induced by starvation. N-SAAW has already been deployed and evaluated in real-world agricultural settings, where it has been shown to identify malnutrition about 3.5 times sooner than current analytical approaches. Furthermore, because of the pre-processing, there was no possibility of the DNN missing starvation.
N-SAAW may be effortlessly linked with farm sensors and provides a way for autonomous monitoring to collect data that is then analyzed to provide insights to farmers. It also includes a visualization and reporting tool, allowing the provenance to be disclosed farther down the supply chain.
The solution developed and demonstrated was a tool to support the monitoring and analysis of milk cows’ bio parameters (temperature, pH). The data on temperature and pH are gathered from a ruminal probe.
The tool’s core functionality is based on a Recurrent Neural Network trained to predict the possible health deterioration of a specimen. In case of a predicted threat, an alert is triggered to inform the farm manager about the malnutrition of an animal. The triggering value has been set to a pH = 5.8, sustaining over a period of a few hours. Figure 1 presents the tool’s user interface, showing measurements and predictions tables, alerts, and pH values over time.
DAC conducted an initial validation of the health deterioration (malnutrition) prediction model. The results of the validation are considered satisfactory. As shown in the confusion matrix below (Figure N), the accuracy of “bad” and “good” predictions is reliable, producing an insignificant number of false positives. The mismatch between “warning” and “good” can be reduced with an auxiliary analytical algorithm.
Figure 2: Confusion matrix of the Recurrent Neural Network (RNN) trained to predict possible health deterioration resulting from malnutrition.
The global streaming market is projected to reach USD 1.6 Trillion by 2029, and the Stream Processing Engine was developed at DAC.Digital is all set to be an integral part of this growth. It was built by our engineers in cooperation with the University of West Bohemia and ICPS (Lativa). The Stream Processing Engine is a platform based on Kafka. The applied lambda architecture enables scalable integration of multiple sensor data and supports real-time and batched data stream processing.
It has already been tested and deployed for stream processing (incoming data as they are measured) and batched data processing (a lot of data is incoming in batches, e.g., once a day). One of the successful deployments was in the dairy industry, where the incoming data from cows’ collars connected to ruminal probes and the data from milking robots were processed.
A more advanced example of using SPE is the practical implementation of Artificial Intelligence algorithms. Data from a topic can provide sets of data to the training algorithms. It might be implemented as a microservice, and by continuous processing of the incoming data, an improvement of the trained algorithm might be achieved. All of these operate and are coordinated through the appropriate implementation within a data stream.
The evolution of the network and sensor network technologies have enabled easy access to real-world information in real time. However, there is still a wide scope to push the envelope in terms of bridging the gap between the requirements of Industry 4.0 and the existing capabilities of stream processing as well as analytics.
Since the beginning of the 20th century, data production has increased exponentially. In recent years there has also been a steep increase in streaming data, which has created a need for more efficient management and utilization of it. In the context of the Internet of Things (IoT), the multiplication of data stream sources (connected devices, sensor networks, etc) has been trending, especially in the cloud.
Concept picture
Researchers have pointed out that although there are several frameworks for stream processing, there is a void regarding the available stream processing platforms/engines that businesses can readily utilize to take advantage of real-world, real-time big data. There is a high dependence on open-source stream processing platforms and documentation and source code available freely. To use this open-source solution, high technical expertise is required, which is scarce as well as expensive.
There is also a need for a solution such as SPE that would be able to integrate multiple streams. Having all electronic devices with their own software makes it impossible to integrate with devices from other companies. It results in a major disadvantage of managing many different software apps to display and process data.
The SPE was developed by DAC.digital, addresses this gap in the state-of-the-art and provides businesses with an easy-to-use emerging technology tool, which could boost the businesses’ operations and productivity.
The SPE provides real-time data analytics based on Lambda Architecture, i.e. a generic, scalable, and fault-tolerant data processing architecture. This architecture is based on an append-only and immutable data source. Thus the serving layer is decoupled from data (events) storage and processing. Figure 1 shows the SPE within the AFarCloud platform Semantic Middleware (High-Level Services layer).
Figure 1: Stream Processing Engine within the AFarCloud platform Semantic Middleware
The aim is to process the data inbound from third-party artifacts (data sources, software systems, and devices such as sensors) within real-time constraints. An example of processing functionalities is as under.
SPE utilizes the Apache Kafka platform to implement a Data Broker, as shown in the figure 1 above. This is the core element that manages the inbound data. Kafka provides tools for managing real-time data pipelines and creating provider, consumer, and streaming applications.
Figure 2 presents SPE within the AFarCloud architecture. It is a part of the Data Management layer. The data is directly provided from the AFarCloud Interfaces layer via the Data Access Manager. Therefore there is no need for any additional middleware component (converter or adaptor) for stakeholders that publish the data within the AFarCloud ecosystem. In order to forward data to the SPE as well, the SPE Data Provider must be implemented.
Figure 2: Stream Processing Engine within the AFarCloud architecture
| SPE based on Kafka | REST | MQTT | |
| Functionality | |||
| Filtration and processing data | allowed on stream | everything must be implemented from scratch | no |
| Requests service | flexible scalability, big throughput | overload by many requests | small throughput |
| Architecture | |||
| Complexity | A complete and standalone application. Use TCP binary protocol for communication. | API exposing endpoints. Use HTTP protocol which is significantly slower than plain TCP binary. | Lightweight transmission protocol. Optimized for sensor networks and M2M. |
| Persistence and reliability | Ensures high reliability by adding the persistence layer and holding copies of streams. | Everything must be implemented from scratch. | No embedded persistence. |
Individual farmers who want their own tailor-made software with the possibility to further extend
Data analysts working in the agricultural field – for rapid development and tests of data processing algorithms in the same framework
SPE can be implemented by an integration/software enterprise as a core used for integrating a few separate agricultural apps into one software or system
Use Case.
Farm dangerous events detection
Scenario.
Sensors that monitor the farm ecosystem (plantation or animal breeding) produce data that can be used in the SPE to detect dangerous events.
Solution.
Stream Processor (see Figure 1) can be used to monitor the specific type of events originating from the farm ecosystem in order to check the defined business rules. In the case of fulfilling the condition, it triggers specific actions, for example, append an alert to the specific topic in the SPE stream. The alert can be handled by dedicated software, for example, a system that informs the end-user (farmer) about the dangerous situation or a system that manages the dedicated vehicle (e.g. drone) to start the mission.
Use Case.
Monitoring of cow breeding zone
Scenario.
Nowadays, modern farm is equipped with sensors that monitor the cow farming ecosystem. Data from these devices should be fused for further processing. This employs applications and algorithms that provide all necessary information about breeding. This approach enables detecting abnormalities, planning future breeding, calculating the costs of infrastructure and herd maintenance, etc.
Solution.
Farm Management System layer (e.g. Decision Support System) or third-party software can consume data from the SPE in order to use it in the breeding domain algorithms and applications that monitor and manage the cow breeding zone.
Our Client
Challenge
Previously, confirmation of the number of litres of milk collected from the producer was entered manually on an External Receipt (PZ) printout. After completing the milk collection route, the drivers would deliver the forms to the Purchasing Department, which had to manually enter more than 2,000 collections per day into the accounting system, and errors arising in the process were verified manually.
Solution
The solution to this problem is the D_Box, a device that automatically records information on the quantity of milk transported and sends it to a cloud platform. The D_Box is a telematic control unit fully integrated into the transport vehicles. Through the D_Box, data from sensors placed on the transport vehicles (such as the milk quantity counter, its temperature or the vehicle’s location) are collected in real time and sent to the cloud platform.
The aim of the project was to complement the IT system on offer with the devices necessary for the digitalisation of the dairy industry. Mobile IoT devices were implemented to monitor in real time the parameters of the goods being transported and the condition of the vehicles themselves, mounted in the transport vehicles. The D_Box enables the automation of data reporting processes in the milk haulage process.
The first versions of the device were installed in around 100 vehicles monitored by our IT system, enabling daily reporting of 2 million litres of milk from 2,000 suppliers. The information gathered allowed us to develop and launch in 2020 an industrial-grade device with an open-source operating system (allowing the system to be self-optimised to its own needs), based on the latest technology, as a response to the needs of the dairy market.
D_Box is a telematics control unit fully integrated with transport vehicles. Through D_Box, data from sensors placed in transport vehicles (such as milk quantity counter, milk temperature or vehicle location) are collected in real time and sent to the cloud platform. Our solution allows you to automate the entire milk collection process. Basing the system on reporting the status and location of transported goods in real time translates into better decision-making, reduced documentation and a significant reduction in errors and abuses. The provided data allows for reducing the number of errors in the recorded number of liters of milk provided by farmers, eliminating the possibility of fraud and forgery during milk collection, automatic control of the entered information and simultaneous reduction of manual data entry, as well as the possibility of dynamically changing the route depending on the needs.
Thanks to the use of a microprocessor in the ARM architecture, D_Box has an operating system that allows its operation without the need to have competences in programming microcontrollers (C language), and the configuration of the device and the creation of your own scripts is done from the operating system level. All input/output terminal elements have drivers, which facilitates their operation in a way similar to communication with a printer or keyboard. Integration with the 4G network and the use of Bluetooth Low Energy (BLE) technology allowed the creation of a fully mobile device intended for transport vehicles. The project uses the latest GSM communication technology. The module works in LTE-M technology, which is dedicated to IoT solutions, including mobile ones, where it is able to maintain the appropriate quality of data transmission even in the case of poor network coverage. D_Box enables the transmission of information from sensors in real time, and thanks to the use of GPS technology, it is possible not only to monitor the parameters of the transported goods, but also their location. Thanks to an extensive communication system based on BLE and Wi-Fi, the devices can be easily configured to work as a master communication device, serving as a medium for data transmission between sensors and the system. Sensors installed in vehicles send data to D_box, and then they go to our IT system. Integration with the Hardware Security Module ensures the highest level of security of the data transmission services provided.
Plug and play approach to IIoT Data Collection
D_Box is an electronic multifunction device for prototyping, MVP development & pilots in Industrial Internet of Things applications. Wireless connection, many interfaces, flexible design.
Single board computer with many applications.
The device is a single-board computer with an ARM processor with a Linux operating system distribution and the peripherals are operated using appropriate commands.
Limitless Implementation
The device can be installed in any place, such as a production hall, farm, warehouse, construction site and many others. It is equipped in GSM module for wireless communication so it is suitable for mobile applications on trucks, construction vehicles, ships etc.
Remote Access
The device receives data from sensors or other devices and sends it using the GSM network to supervisory systems. It provides remote access to data from devices installed onsite. D_Box enables over-the-air updates and remote configuration of sensors.
Validated Technology
We have implemented D_Box has into a popular IT system for the dairy industry. It allowed automating the entire milk collection process, eliminating manual data entry about amounts of milk provided by farmers, transport optimization, and monitoring.
Hardware for special tasks
D_Box enables wireless communication via GSM, BLE and Wi-Fi networks and wired communication – CAN bus, USB, Ethernet, I2C. Selection of external connectors, customized mounting options, custom branding and many more options are available to meet your needs.
With the introduction of D_Box, the company has complemented the range of its own IT products with a multi-purpose hardware solution that provides the necessary real-time data for the entire milk collection process and allows it to be automated. By integrating our software system with a dedicated hardware solution, we were able to offer the dairy industry a comprehensive solution to meet its needs.
Due to its versatility, we also dedicate the device to a broader spectrum of companies related to the IoT industry or companies looking to develop their product. The D_Box provides the professional, multi-tasking hardware layer necessary for this type of project. It is the only such solution dedicated to IT companies that they can use during their own IoT projects. The company plans to attract partners interested in purchasing the devices for their own proprietary solutions as a finished hardware component.
