{"id":25530,"date":"2025-01-13T13:00:00","date_gmt":"2025-01-13T13:00:00","guid":{"rendered":"https:\/\/pl.server2.purelogics.net\/?p=25530"},"modified":"2025-01-13T08:16:57","modified_gmt":"2025-01-13T08:16:57","slug":"in-stream-inference-architecture-with-sagemaker-apache-flink-aws-kinesis","status":"publish","type":"post","link":"https:\/\/pl.server2.purelogics.net\/?p=25530","title":{"rendered":"Real-Time In-Stream Inference Architecture with SageMaker, Apache Flink, &amp; AWS Kinesis"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_65 counter-hierarchy ez-toc-counter ez-toc-light-blue ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title \" >Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\/#Real-Time_In-Stream_Inference_Architecture\" title=\"Real-Time In-Stream Inference Architecture&nbsp;\">Real-Time In-Stream Inference Architecture&nbsp;<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\/#Revolutionize_Real-Time_Data\" title=\"Revolutionize Real-Time Data\">Revolutionize Real-Time Data<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\/#Apache_Flink\" title=\"Apache Flink&nbsp;\">Apache Flink&nbsp;<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\/#Apache_Flink_and_KDA\" title=\"Apache Flink and KDA\">Apache Flink and KDA<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\/#Asynchronous_IO_Operator_of_Flink\" title=\"Asynchronous I\/O Operator of Flink\">Asynchronous I\/O Operator of Flink<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\/#Where_does_Amazon_SageMaker_Stand\" title=\"Where does Amazon SageMaker Stand\">Where does Amazon SageMaker Stand<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\/#Getting_Started_with_AWS\" title=\"Getting Started with AWS\">Getting Started with AWS<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\/#Conclusion\" title=\"Conclusion&nbsp;\">Conclusion&nbsp;<\/a><\/li><\/ul><\/nav><\/div>\n<span class=\"rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\">4<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>\n<p class=\"wp-block-paragraph\">As organizations rush toward digital transformation, the challenge is to handle large volumes of data, and the worth of that information fades over time. The difficulty is to evaluate, learn, and interpret real-time information to detect anomalies, get accurate outcomes, and predict future states.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We\u2019ll talk about the architecture for a service that can get real-time inference on any streaming data\u2014furthermore, the Amazon Kinesis Data Analytics with Apache Flink to independently evoke any underlying databases or services.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In-stream data that is managed in real-time is hard to understand, so let\u2019s break it down into two categories:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Real-time inference <\/strong>is the capability to utilize information from the feed to the future state of the project for the fundamental data.\u00a0<\/li>\n\n\n\n<li><strong>Instream data <\/strong>means the ability to process a stream of data that gathers, analyzes, and processes information.\u00a0<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Assume a streaming app that gathers credit card transactions with a few other parameters, including source IP, to get the geographic details of the amount and transaction. This information is later used to conclude fraudulent transactions immediately if you compare this to a traditional batch-oriented approach that finds fraudulent transactions after the end of the business day and produces a report when time has passed.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Real-Time_In-Stream_Inference_Architecture\"><\/span>Real-Time In-Stream Inference Architecture&nbsp;<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Let\u2019s talk about leveraging Amazon Kinesis Data Analytics for Apache Flink (KDA), Amazon SageMaker, Apache Flink, and Amazon API Gateway to tackle challenges like real-time fraud detection on streaming credit card transactions. We outline how to build a reliable, scalable, and highly available streaming architecture using managed services, significantly reducing operational overhead compared to managing your infrastructure. We focus on setting up and executing Flink applications using KDA for Apache Flink.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The data is fed to the Kinesis Data Streams utilizing the Amazon Kinesis Producer Library in the architecture. You can use any KDS-supported ingestion pattern. KDS streams this data to an Apache Flink-based KDA application, which manages the necessary infrastructure, scales based on traffic patterns, and automatically handles failure recovery. The Flink application is set up to call an API Gateway endpoint using Asynchronous I\/O. This API Gateway is connected to an AWS SageMaker endpoint, though you could use any suitable endpoint for your data enrichment needs. Flink distributes the data across multiple stream partitions, and user-defined operators can transform the data stream.<\/p>\n\n\n\n<div class=\"cta-section\" style=\"background-image: url('https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2024\/05\/image_2024_05_27T12_40_05_229Z.jpg'); padding: 30px; border-radius: 15px; color: white; background-size: cover; background-position: center;\">\n  <div style=\"text-align: left;\">\n    <h3 style=\"margin-top: 0; color: white;\"><span class=\"ez-toc-section\" id=\"Revolutionize_Real-Time_Data\"><\/span>Revolutionize Real-Time Data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n    <p>Use the power of SageMaker, Flink, and Kinesis for seamless insights.<\/p>\n  <\/div>\n  <div style=\"text-align: left;\">\n    <a href=\"javascript:void(0);\"class=\"cta-button\">Let\u2019s build smarter solutions together!\n      <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 24 24\" width=\"16\" height=\"16\" fill=\"white\" class=\"arrow-icon\"><path d=\"M12 2L10.59 3.41 17.17 10H2v2h15.17l-6.58 6.59L12 22l10-10L12 2z\"\/><\/svg>\n    <\/a>\n  <\/div>\n<\/div>\n\n<style>\nbody, .cta-section {\n  font-family: 'Sen', sans-serif;\n}\n\n.cta-section {\n  background-size: cover;\n  background-position: center;\n  border-radius: 15px;\n  padding: 50px;\n  color: white;\n  display: block; \n}\n.cta-section h2 {\n  font-size: 2.5em;\n  margin-bottom: 5px !important; \n  color: white;\n}\n.cta-section p {                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        margin-bottom: 30px !important; \n}\n.cta-button {\n  display: inline-flex; \n  align-items: center; \n  padding: 8px 16px;\n  font-size: 0.8em;\n  color: white !important; \n  background-color: #4aba6a;\n  text-decoration: none;\n  border-radius: 10px;\n  position: relative;\n}\n.cta-button .arrow-icon {\n  margin-left: 10px; \n  vertical-align: middle; \n}\n.cta-button:hover {\n  background-color: #3b8e54;\n}\n<\/style>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Apache_Flink\"><\/span>Apache Flink&nbsp;<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This is an open-source distributed processing framework built for stateful computations on both bounded and unbounded datasets. This architecture utilizes KDA and Apache Flink for in-stream analytics and employs an Asynchronous I\/O operator to interact with external systems.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Apache_Flink_and_KDA\"><\/span>Apache Flink and KDA<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">KDA for Apache Flink is a fully managed <a href=\"https:\/\/pl.server2.purelogics.net\/amazon-services\/\">AWS service<\/a> that allows you to use Apache Flink for processing streaming data. You can use Scala and Java with KDA to examine data streams. The service handles the underlying infrastructure, including tasks like provisioning compute resources, parallel computation, scaling, and backing up your applications via checkpoints and snapshots.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Asynchronous_IO_Operator_of_Flink\"><\/span>Asynchronous I\/O Operator of Flink<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"614\" src=\"https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2025\/01\/Inside-2-1024x614.jpg\" alt=\"sagemaker\" class=\"wp-image-25531\" srcset=\"https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2025\/01\/Inside-2-1024x614.jpg 1024w, https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2025\/01\/Inside-2-300x180.jpg 300w, https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2025\/01\/Inside-2.jpg 1200w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The asynchronous I\/O operator in Flink enables interaction with external systems to enrich stream events or perform computations asynchronously. This operator allows a single parallel function to handle multiple requests and responses simultaneously, often boosting streaming throughput. The Asynchronous I\/O API integrates smoothly with data streams, handling concerns like order, event time, and fault tolerance. You can configure it to call external databases or APIs, like the architecture that uses API Gateway integrated with SageMaker endpoints.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Where_does_Amazon_SageMaker_Stand\"><\/span>Where does Amazon SageMaker Stand<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In this architecture, there\u2019s an Amazon SageMaker endpoint being used to identify fraudulent transactions through API Gateway. Amazon SageMaker is a fully managed service that enables developers and data scientists to build, train, and deploy machine learning (ML) models quickly. SageMaker streamlines the ML process, making it easier to develop high-quality models, which can then be deployed for real-time inferences.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">You can create persistent endpoints with SageMaker to receive predictions from models deployed through SageMaker hosting services.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Getting_Started_with_AWS\"><\/span>Getting Started with AWS<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To help you jumpstart the process, AWS offers the AWS Streaming Data Solution for Amazon Kinesis (Option 4), a pre-built solution available as a one-click CloudFormation template. You can then structure a real-time inference pipeline within a few minutes. While the solution leverages <a href=\"https:\/\/pl.server2.purelogics.net\/aws-lambda-vs-aws-ec2\/\">AWS Lambda<\/a>, it can easily be modified to use SageMaker, replicating the architecture described in this post. There is also an <a href=\"https:\/\/pl.server2.purelogics.net\/amazon-services\/\">AWS Solutions <\/a>Construct that connects an API Gateway to a SageMaker endpoint, which can replace Lambda in the setup. For more information, check the solution\u2019s implementation guide.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The diagram below showcases the architecture of this solution.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"614\" src=\"https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2025\/01\/Inside-1024x614.jpg\" alt=\"sagemaker\" class=\"wp-image-25533\" srcset=\"https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2025\/01\/Inside-1024x614.jpg 1024w, https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2025\/01\/Inside-300x180.jpg 300w, https:\/\/pl.server2.purelogics.net\/wp-content\/uploads\/2025\/01\/Inside.jpg 1200w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion&nbsp;<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In this post, we outlined an architecture that uses Kinesis Data Streams (KDS), Kinesis Data Analytics for Apache Flink (KDA), Apache Flink, and Amazon SageMaker to build a highly available and scalable real-time inference application. Using managed services like these means you don\u2019t need to worry about provisioning or maintaining infrastructure\u2014letting you focus on deriving insights and running real-time inferences from your data streams.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/pl.server2.purelogics.net\/amazon-services\/\">AWS Streaming Data Solution<\/a> for Amazon Kinesis offers a better way to automatically implement the key AWS solutions to find, store, process, and infer from streaming data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">PureLogics has in offering manageable and scalable AWS cloud solutions. <a href=\"https:\/\/pl.server2.purelogics.net\/contact-us\/\">Contact us today<\/a>!\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\">4<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span> As organizations rush toward digital transformation, the challenge is to handle large volumes of data, and the worth of that information fades over time. The difficulty is to evaluate, learn, and interpret real-time information to detect anomalies, get accurate outcomes, and predict future states.&nbsp; We\u2019ll talk about the architecture for a service that can get <a href=\"https:\/\/pl.server2.purelogics.net\/?p=25530\" class=\"more-link\">&#8230;<\/a><\/p>\n","protected":false},"author":11,"featured_media":25534,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[13],"tags":[],"class_list":["post-25530","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud-technologies"],"_links":{"self":[{"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=\/wp\/v2\/posts\/25530","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=25530"}],"version-history":[{"count":1,"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=\/wp\/v2\/posts\/25530\/revisions"}],"predecessor-version":[{"id":25535,"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=\/wp\/v2\/posts\/25530\/revisions\/25535"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=\/wp\/v2\/media\/25534"}],"wp:attachment":[{"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=25530"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=25530"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pl.server2.purelogics.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=25530"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}