Example: FPGA coprocessor accelerated Ingest
In this example telematic data of a complex type is produced in many cars, trucks, trailers, etc. moving throughout the world. The interested party wishes to execute analytics based on a stream of updates flowing from a global base of registered vehicles. Data is generated within the vehicle on a CAN bus and is converted in-situ to JSON code that acts as our source for data ingest.
The Sigmax Stack includes FPGA hardware assisted conversion to Apache Arrow. Converted data are pushed from the FPGA smartNIC to an Apache Pulsar message broker. This broker has the benefit of being natively higher throughput than any other open source message broker. It further benefits from underlying Intel OPTANE support to further increase its maximum throughput and widen its recent data processing window. The entire solution scales perfectly linearly to data ingest and higher data throughput means less hardware and cost in your analytics pipeline. All of these elements tie neatly together and natively support the persistent storage and support aging data into very cost effective cloud (s3 for example) storage.