Connect with us

AI

Model serving in Java with AWS Elastic Beanstalk made easy with Deep Java Library

Avatar

Published

on

Deploying your machine learning (ML) models to run on a REST endpoint has never been easier. Using AWS Elastic Beanstalk and Amazon Elastic Compute Cloud (Amazon EC2) to host your endpoint and Deep Java Library (DJL) to load your deep learning models for inference makes the model deployment process extremely easy to set up. Setting up a model on Elastic Beanstalk is great if you require fast response times on all your inference calls. In this post, we cover deploying a model on Elastic Beanstalk using DJL and sending an image through a post call to get inference results on what the image contains.

About DJL

DJL is a deep learning framework written in Java that supports training and inference. DJL is built on top of modern deep learning engines (such as TenserFlow, PyTorch, and MXNet). You can easily use DJL to train your model or deploy your favorite models from a variety of engines without any additional conversion. It contains a powerful model zoo design that allows you to manage trained models and load them in a single line. The built-in model zoo currently supports more than 70 pre-trained and ready-to-use models from GluonCV, HuggingFace, TorchHub, and Keras.

Benefits

The primary benefit of hosting your model using Elastic Beanstalk and DJL is that it’s very easy to set up and provides consistent sub-second responses to a post request. With DJL, you don’t need to download any other libraries or worry about importing dependencies for your chosen deep learning framework. Using Elastic Beanstalk has two advantages:

  • No cold startup – Compared to an AWS Lambda solution, the EC2 instance is running all the time, so any call to your endpoint runs instantly and there isn’t any ovdeeerhead when starting up new containers.
  • Scalable – Compared to a server-based solution, you can allow Elastic Beanstalk to scale horizontally.

Configurations

You need to have the following gradle dependencies set up to run our PyTorch model:

plugins { id 'org.springframework.boot' version '2.3.0.RELEASE' id 'io.spring.dependency-management' version '1.0.9.RELEASE' id 'java'
} dependencies { implementation platform("ai.djl:bom:0.8.0") implementation "ai.djl.pytorch:pytorch-model-zoo" implementation "ai.djl.pytorch:pytorch-native-auto" implementation "org.springframework.boot:spring-boot-starter" implementation "org.springframework.boot:spring-boot-starter-web"
}

The code

We first create a RESTful endpoint using Java SpringBoot and have it accept an image request. We decode the image and turn it into an Image object to pass into our model. The model is autowired by the Spring framework by calling the model() method. For simplicity, we create the predictor object on each request, where we pass our image for inference (you can optimize this by using an object pool) . When inference is complete, we return the results to the requester. See the following code:

 @Autowired ZooModel<Image, Classifications> model; /** * This method is the REST endpoint where the user can post their images * to run inference against a model of their choice using DJL. * * @param input the request body containing the image * @return returns the top 3 probable items from the model output * @throws IOException if failed read HTTP request */ @PostMapping(value = "/doodle") public String handleRequest(InputStream input) throws IOException { Image img = ImageFactory.getInstance().fromInputStream(input); try (Predictor<Image, Classifications> predictor = model.newPredictor()) { Classifications classifications = predictor.predict(img); return GSON.toJson(classifications.topK(3)) + System.lineSeparator(); } catch (RuntimeException | TranslateException e) { logger.error("", e); Map<String, String> error = new ConcurrentHashMap<>(); error.put("status", "Invoke failed: " + e.toString()); return GSON.toJson(error) + System.lineSeparator(); } } @Bean public ZooModel<Image, Classifications> model() throws ModelException, IOException { Translator<Image, Classifications> translator = ImageClassificationTranslator.builder() .optFlag(Image.Flag.GRAYSCALE) .setPipeline(new Pipeline(new ToTensor())) .optApplySoftmax(true) .build(); Criteria<Image, Classifications> criteria = Criteria.builder() .setTypes(Image.class, Classifications.class) .optModelUrls(MODEL_URL) .optTranslator(translator) .build(); return ModelZoo.loadModel(criteria); } 

A full copy of the code is available on the GitHub repo.

Building your JAR file

Go into the beanstalk-model-serving directory and enter the following code:

cd beanstalk-model-serving
./gradlew build

This creates a JAR file found in build/libs/beanstalk-model-serving-0.0.1-SNAPSHOT.jar

Deploying to Elastic Beanstalk

To deploy this model, complete the following steps:

  1. On the Elastic Beanstalk console, create a new environment.
  2. For our use case, we name the environment DJL-Demo.
  3. For Platform, select Managed platform.
  4. For Platform settings, choose Java 8 and the appropriate branch and version.

  1. When selecting your application code, choose Choose file and upload the beanstalk-model-serving-0.0.1-SNAPSHOT.jar that was created in your build.
  2. Choose Create environment.

After Elastic Beanstalk creates the environment, we need to update the Software and Capacity boxes in our configuration, located on the Configuration overview page.

  1. For the Software configuration, we add an additional setting in the Environment Properties section with the name SERVER_PORT and value 5000.
  2. For the Capacity configuration, we change the instance type to t2.small to give our endpoint a little more compute and memory.
  3. Choose Apply configuration and wait for your endpoint to update.

Calling your endpoint

Now we can call our Elastic Beanstalk endpoint with our image of a smiley face.

See the following code:

curl -X POST -T smiley.png <endpoint>.elasticbeanstalk.com/inference

We get the following response:

[ { "className": "smiley_face", "probability": 0.9874626994132996 }, { "className": "face", "probability": 0.004804758355021477 }, { "className": "mouth", "probability": 0.0015588520327582955 }
]

The output predicts that a smiley face is the most probable item in our image. Success!

Limitations

If your model isn’t called often and there isn’t a requirement for fast inference, we recommend deploying your models on a serverless service such as Lambda. However, this adds overhead due to the cold startup nature of the service. Hosting your models through Elastic Beanstalk may be slightly more expensive because the EC2 instance runs 24 hours a day, so you pay for the service even when you’re not using it. However, if you expect a lot of inference requests a month, we have found the cost of model serving on Lambda is equal to the cost of Elastic Beanstalk using a t3.small when there are about 2.57 million inference requests to the endpoint.

Conclusion

In this post, we demonstrated how to start deploying and serving your deep learning models using Elastic Beanstalk and DJL. You just need to set up your endpoint with Java Spring, build your JAR file, upload that file to Elastic Beanstalk, update some configurations, and it’s deployed!

We also discussed some of the pros and cons of this deployment process, namely that it’s ideal if you need fast inference calls, but the cost is higher when compared to hosting it on a serverless endpoint with lower utilization.

This demo is available in full in the DJL demo GitHub repo. You can also find other examples of serving models with DJL across different JVM tools like Spark and AWS products like Lambda. Whatever your requirements, there is an option for you.

Follow our GitHub, demo repository, Slack channel, and Twitter for more documentation and examples of DJL!


About the Author

Frank Liu is a Software Engineer for AWS Deep Learning. He focuses on building innovative deep learning tools for software engineers and scientists. In his spare time, he enjoys hiking with friends and family.

Source: https://aws.amazon.com/blogs/machine-learning/model-serving-in-java-with-aws-elastic-beanstalk-made-easy-with-deep-java-library/

AI

6 Best Artificial Intelligence Apps Making Human Lives Easier

Avatar

Published

on

It is no secret that Artificial Intelligence (AI) has gained a lot of popularity and taken the world by storm. Past studies have revealed that the global market of AI is predicted to reach from US$29.82 billion in 2019 to US$252.01 billion in 2025, growing at a CAGR of 42.7 percent.

“Research by Gartner identified that in the next few years, every application or service will incorporate AI at some level. ” 

Due to its benefits, AI is being used in different business functions and fields, including marketing, operations, finance, etc. 

Besides this, AI allows computers to extract valuable insights from large volumes of data, thereby helping perform complex tasks with ease. Moreover, AI-based apps have not only facilitated organizations, but have made human lives a lot easier. For example, rule-based automation and machine learning have simplified many routine jobs.   

Apart from business applications, AI has now gained popularity in the real world scenario and is being increasingly used in web-based and mobile applications. And for this reason, AI-enabled apps are being increasingly embedded in mobile operating systems that are available in the market.

We have created a list of the 6 best Artificial Intelligence apps that are making human lives easier. 

#1 Google Assistant 

The first one on our list is none other than the famous Google Assistant, which is an AI-driven virtual app generally available on smart home devices and mobile phones. The good thing about this AI app is that it supports different languages. This gives users a chance to interact with the app by using their mother tongue. 

Apart from this, the Google Assistant can perform all the possible tasks that can make your life easy and hassle-free. Not sure which ones? Have a look at the list below:

  • It can access the required information from your calendar. 
  • It can look up for information online, such as restaurant bookings, weather updates, etc. 
  • It can play your favorite music for you. 
  • It facilitates in making appointments and sending messages. 
  • It can open different apps on your phone or other smart devices. 

#2 Ada – Healthcare in Your Hand

Ada is another AI-based app that has a clean and interactive user interface and it helps you in understanding what are you suffering from. This app works like any other doctor’s appointment as you are required to share your basic details first and then details regarding the symptoms.

Based on the information that you provide, it prepares a medical report regarding your current condition, which includes a summary, potential causes, etc. You can also share that report as a PDF with your doctor or any medical expert for further diagnosis. 

Besides this, Ada is an accessible health advisor which is powered by a smart Artificial Intelligence engine. The good thing about this app is that it considers all important patient information, such as past medical history, risk factors, and more. And after gathering this data, it leverages machine learning and AI to provide accurate feedback.

#3 Hookt

Another one on our list is Hookt which is a platform that facilitates in chatting, making friends with people who share common interests. So, if you are somebody who is fond of making new friends in your area or while traveling, you should try the Hookt app. 

All in all, it is crucial to note that sometimes apps ask you to share your personal information, thus increasing the likelihood of scams. A study carried out by Business Insider revealed that 1 out of every 10 adults falls prey to a scam each year in the US. 

Therefore, you should be well aware of how to detect a fraud or scam as this can save your money. For instance, previously, many Vodafone users reported that they had to pay for the airG chat service even when they didn’t subscribe for it. However, airG was fast enough to identify this airG spam and took corrective actions immediately. Since not everyone is as responsible and authentic as airG, you should be beware. 

#4 Socratic 

AI and Robotics have not only revolutionized the way businesses are run, instead they are helping improve educational standards at schools considerably.  

“Socratic is one of the most popular AI-based apps for iOS and Android that helps students with their homework.” 

This app provides educational and informative resources, such as explanatory videos, definitions, etc. To use this app, users are required to take a photo of their homework, and then the app provides them with solutions and principles that can help solve the problems. 

The best thing about this app is that it is not subject-specific, instead, it provides help regarding different subjects, including Chemistry, English, Economics, Math, and Science. 

#5 YVA

YVA is a cloud-based app that is widely used in many different organizations. This app leverages AI for evaluating the performance of employees. Apart from this, YVA can be connected to instant messengers and corporate mail, thereby making it a one-stop solution. 

Another good thing about this app is that it conducts regular employee surveys, and analyzes the received information. This helps send the required warnings to employees and managers. Besides this, YVA helps prevent conflicts in teams, as it recognizes the competencies, like strengths, weaknesses, and leadership qualities of each employee. 

#6 Juke Deck

Juke Deck leverages the full potential of AI-based technologies for creating music tracks of different genres. However, the users of this app are required to identify the initial parameters of the composition, including the genre, tempo, duration, mood, etc. Once these variables are finalized, the Create Track Buttons helps process the track. 

The good thing is that you can either listen to the music that you have composed in a browser or download it on your computer. Revisions to the processed track can be done easily as well. Apart from this, Juke Deck does not ask for royalties, therefore you can use the work created as you wish. For example, you can post your created tracks on social media or YouTube. 

The Final Words 

In a nutshell, Artificial Intelligence has completely revolutionized the way we live and has made our lives a lot more convenient, on both – a personal and professional level. Therefore, you should try the above-mentioned AI-powered apps and enjoy their benefits.

The post 6 Best Artificial Intelligence Apps Making Human Lives Easier appeared first on Aiiot Talk – Artificial Intelligence | Internet of Things | Technology.

Checkout PrimeXBT
Trade with the Official CFD Partners of AC Milan
Source: http://www.aiiottalk.com/artificial-intelligence-apps-making-human-lives-easier/

Continue Reading

Aerospace

We Need to Build a Cloud for the Next Decade: Satya Nadella

Avatar

Published

on

As the world enters the second wave of digital transformation, we need to foundationally transform how the cloud can drive the next level of economic growth, Microsoft Corp. Chief Executive Satya Nadella said.

Nadella was speaking at the Microsoft Ignite 2021, the annual flagship event virtually attended by more than 100,000 global participants, including IT decision-makers, developers, data professionals, security professionals and other technology enthusiasts.

“The true test of technology has always been whether organizations can improve their time to value, increase agility, and reduce costs…but as the world recovers, it will require much more from technology and the cloud, in particular, to help address our most pressing challenges,” Nadella said.

Five key attributes will drive the next-generation of innovation in the cloud, Nadella said. “These are ubiquitous and decentralized computing, sovereign data and ambient intelligence, empowered creators and communities, expanded economic opportunity for the global workforce, and trust by design,” he said.

As businesses have accelerated their digital journeys over the past year, the demand for technology has significantly picked up to enable various use-cases like telehealth, remote manufacturing, and new ways of working from home. Microsoft believes the cloud has been the foundation to enable all of these.

Microsoft announced ‘Microsoft Mesh’, a new mixed reality platform built on, which enables geographically distributed teams to interact holographically with each other.

“With mixed reality technology, the digital and physical worlds have come together,” Nadella said.

“This has been the dream for mixed reality, the idea from the very beginning,” said Microsoft Technical Fellow Alex Kipman. “You can actually feel like you are in the same place with someone sharing content or you can teleport from different mixed reality devices and be present with people even when you’re not physically together.”

Microsoft also announced a new semantic search capability in Azure Cognitive Search, an Artificial Intelligence (AI)-powered cloud search service for mobile and web app development. This capability enables developers to deliver results based on user intent as opposed to a keyword-based search, which is the industry norm. Semantic search leverages some of the most advanced natural language models to improve the relevance and ranking of search results.

The post We Need to Build a Cloud for the Next Decade: Satya Nadella appeared first on ELE Times.

Checkout PrimeXBT
Trade with the Official CFD Partners of AC Milan
Source: https://www.eletimes.com/we-need-to-build-a-cloud-for-the-next-decade-satya-nadella

Continue Reading

Artificial Intelligence

AI Offers Powerful Insights into Video Content Production

Avatar

Published

on

AI, language processing, and deep learning are three major technologies that affect streaming services. They are already used to create content and the picture looks beautiful and detailed. If we exploit Artificial Intelligence in live streaming, this would change everything. Modern technology allows us to create automated actions, trigger real-time graphics, analyze audio, monitor social media sentiment and auto-sharing.

AI Leads to New Opportunities for Video Content Development

Some developers saw AI technology as an opportunity and created useful tools that help in video production. Let’s talk about 4 software variants that could be used for video production.

NotMP3

To process multimedia you should have a useful tool that will allow getting them without payment. Nowadays the best content provider is Youtube. That is why we decided to mention NotMP3 software, which uses cutting-edge AI technology. This sort will allow you to fulfill this task for free.

This is free software which is downloadable from the site. This is a simple but very powerful tool that can help you to get audio or video file of any quality you like. You can also get files from Soundcloud and transfer them to various formats.

The main advantage of NotMP3 software is that it is completely free and does not require creating an account. All you have to do is to install and use it.  

Transcriptive AI

The first program that actually uses AI technology is Transcriptive AI. This is a universal tool that is exploited to transcribe footage. It will make that footage searchable. You can use this software to make caption and paper edits.

The creator allows getting a free trial copy but if you want to continue using it, you will have to buy the product. This soft will be useful if you want to find some pieces of audio that have been mentioned in the video. You will also have accurate transcripts spending as little time as possible. The most powerful component is searchability – you can look for clips, sequences, and markers. This machine learning tool works like Google and you just mention the keywords.

Pixop

This is not a tool but a cloud service that exploits intuitive interface and AI technologies for video enhancement. The main advantage of this service is that it is very easy to use. You will not need any plugins and other tools. No subscription and fees are required. This service has been created to encourage creators to update their digital archives as easily as it is possible.

Users value Pixop because it provides a place where you can make video enhancement as fast as possible. This is a flexible and scalable could service which has been created to tackle every requirement. And the most important feature is the fact that it is powered by machine learning – just let AI do the work! Just sign up and use the online tool.

Imagen

This is a media management tool that is used for keeping media safe, secure, and easy to find. This program will help you to organize and unlock the full potential of your video files. You just open a video file and start editing it. This software uses a simple but effective technology to process a picture. Choose a frame, pause it and crop out the needed fragment. Then this picture could be saved on your device and used to create media products. You can add annotations for any image.

This tool could be ideal for content makers that are willing to organize and see the true value of their content. This service lets you focus on the important features of the picture. It is very good for business in content-making industry.

AI is Invaluable for Creating New Video Content

So, what is the best AI solution for content creation? At first, you will need a tool that would be able to download the high-quality video as fast as possible. That is why you will definitely need NotMP3 downloader and converter.

After that try Transcriptive AI program because this is professional software. It has lots of useful features. But it is not free.

If you are looking for the fastest way, try Pixop cloud service – it has all the needed functions. You can use Imagen as a free offline variant which will help you to create the best content.

The post AI Offers Powerful Insights into Video Content Production appeared first on SmartData Collective.

Checkout PrimeXBT
Source: https://www.smartdatacollective.com/ai-offers-insights-into-video-content-production/

Continue Reading

Artificial Intelligence

AI Offers Powerful Insights into Video Content Production

Avatar

Published

on

AI, language processing, and deep learning are three major technologies that affect streaming services. They are already used to create content and the picture looks beautiful and detailed. If we exploit Artificial Intelligence in live streaming, this would change everything. Modern technology allows us to create automated actions, trigger real-time graphics, analyze audio, monitor social media sentiment and auto-sharing.

AI Leads to New Opportunities for Video Content Development

Some developers saw AI technology as an opportunity and created useful tools that help in video production. Let’s talk about 4 software variants that could be used for video production.

NotMP3

To process multimedia you should have a useful tool that will allow getting them without payment. Nowadays the best content provider is Youtube. That is why we decided to mention NotMP3 software, which uses cutting-edge AI technology. This sort will allow you to fulfill this task for free.

This is free software which is downloadable from the site. This is a simple but very powerful tool that can help you to get audio or video file of any quality you like. You can also get files from Soundcloud and transfer them to various formats.

The main advantage of NotMP3 software is that it is completely free and does not require creating an account. All you have to do is to install and use it.  

Transcriptive AI

The first program that actually uses AI technology is Transcriptive AI. This is a universal tool that is exploited to transcribe footage. It will make that footage searchable. You can use this software to make caption and paper edits.

The creator allows getting a free trial copy but if you want to continue using it, you will have to buy the product. This soft will be useful if you want to find some pieces of audio that have been mentioned in the video. You will also have accurate transcripts spending as little time as possible. The most powerful component is searchability – you can look for clips, sequences, and markers. This machine learning tool works like Google and you just mention the keywords.

Pixop

This is not a tool but a cloud service that exploits intuitive interface and AI technologies for video enhancement. The main advantage of this service is that it is very easy to use. You will not need any plugins and other tools. No subscription and fees are required. This service has been created to encourage creators to update their digital archives as easily as it is possible.

Users value Pixop because it provides a place where you can make video enhancement as fast as possible. This is a flexible and scalable could service which has been created to tackle every requirement. And the most important feature is the fact that it is powered by machine learning – just let AI do the work! Just sign up and use the online tool.

Imagen

This is a media management tool that is used for keeping media safe, secure, and easy to find. This program will help you to organize and unlock the full potential of your video files. You just open a video file and start editing it. This software uses a simple but effective technology to process a picture. Choose a frame, pause it and crop out the needed fragment. Then this picture could be saved on your device and used to create media products. You can add annotations for any image.

This tool could be ideal for content makers that are willing to organize and see the true value of their content. This service lets you focus on the important features of the picture. It is very good for business in content-making industry.

AI is Invaluable for Creating New Video Content

So, what is the best AI solution for content creation? At first, you will need a tool that would be able to download the high-quality video as fast as possible. That is why you will definitely need NotMP3 downloader and converter.

After that try Transcriptive AI program because this is professional software. It has lots of useful features. But it is not free.

If you are looking for the fastest way, try Pixop cloud service – it has all the needed functions. You can use Imagen as a free offline variant which will help you to create the best content.

The post AI Offers Powerful Insights into Video Content Production appeared first on SmartData Collective.

Checkout PrimeXBT
Source: https://www.smartdatacollective.com/ai-offers-insights-into-video-content-production/

Continue Reading
Esports3 days ago

PowerOfEvil on TSM’s Spring Split playoff preparation: ‘A lot of things are going to change in the next couple of days’

AR/VR2 days ago

‘Farpoint’ Studio Impulse Gear Announces a New VR Game Coming This Year

Blockchain22 hours ago

‘Bitcoin Senator’ Lummis Optimistic About Crypto Tax Reform

Blockchain22 hours ago

NEXT Chain: New Generation Blockchain With Eyes on the DeFi Industry

Gaming1 day ago

Betfred Sports, Represented by SCCG Management, Signs Multi-year Marketing Agreement with the Colorado Rockies

Aerospace3 days ago

Astra’s 100-year plan: Q&A with CEO Chris Kemp

Blockchain22 hours ago

Bitcoin Price Analysis: Back Above $50K, But Facing Huge Resistance Now

Blockchain22 hours ago

Dogecoin becomes the most popular cryptocurrency

Esports3 days ago

How to download Pokemon Unite APK, iOS, and Switch

Blockchain22 hours ago

Billionaire Hedge Fund Manager and a Former CFTC Chairman Reportedly Invested in Crypto Firm

Blockchain22 hours ago

Institutional Investors Continue to Buy Bitcoin as Price Tops $50K: Report

Cyber Security3 days ago

How you can get someone’s Snapchat password?

Payments2 days ago

4-parter on Coinbase “IPO” – Part 1 = 5 Reasons Why It Matters

Aerospace2 days ago

Partners produce rotor blade 3D-printed tool on Ingersoll 3D printer

Crowdfunding5 days ago

Verifi Reveals that Nearly $31 Billion Is Lost Yearly to Transaction Disputes, which May Be Reduced via “Proactive Management”

Cyber Security3 days ago

Critical Vulnerability Discovered in a Firewall Appliance Made by Genua

Automotive4 days ago

Rivian shares details on the R1T pickup’s clever battery heating strategies

Crowdfunding5 days ago

Fintech Unicorn Brex Explains how Digital Commerce Startups can Scale Operations by Selling Wholesale

HRTech3 days ago

Only 57% Indian employees feel GTL insurance cover by employer is sufficient

Crowdfunding5 days ago

Kraken Bank CEO David Kinitsky Provides Glimpse into How He’s Planning to Set Up Operations, As Company Might Acquire More Funding

Trending