Connect with us

AI

Michael Hurlston: How Synaptics pivoted from mobile/PC sensors to the internet of things

Avatar

Published

on

Synaptics pioneered sensors for touchscreens for PCs and mobile devices. But the San Jose-based hardware company has shifted to where the processing is happening — at the edge of the network.

Under CEO Michael Hurlston, the 35-year-old company has pivoted away from its early markets and focused on artificial intelligence at the edge to bring greater efficiency to internet of things (IoT) devices. With AI at the edge, the company can process the sensor data that it collects and only send alerts when they’re relevant to the network.

Hurlston said that processing paradigm will offload crowded home and business networks and ensure privacy for customer data that doesn’t have to be stored in big datacenters. In the company’s most recent quarter ended December 31, the internet of things now accounts for 43% of the company’s overall $358 million in quarterly revenue, while the PC is 26% and mobile is 31%. Synaptics has 1,400 employees.

Synaptics’ customers now span consumer, enterprise, service provider, automotive, and industrial markets. IoT markets for chips are expected to grow 10% to 15% a year, and the company recently picked up better wireless chip products from Broadcom. Synaptics also launched its new Katana low-power AI processors for the edge. I spoke with Hurlston, who has been in the top job for 18 months, about this transformation.

Here’s an edited transcript of our interview.

Above: Michael Hurlston is CEO of Synaptics.

Image Credit: Synaptics

Michael Hurlston: You understand the business probably better than most. We’ve been thought of as mobile, mobile, mobile, and then maybe a PC subhead. We’ve tried to move the company into IoT, and then mobile where we can attack opportunistically. We’re trying to make IoT our big thrust. That’s what came out this quarter. IoT was our largest business. People started believing that we could make that happen. That’s the main thing.

VentureBeat: What do you think about the future in terms of haptics and the sense of touch? I’m a science fiction fan, and I just got done with the latest Ready Player Two book. They had a VR system in there that could reproduce all of the senses for you.

Hurlston: It sounds both interesting and dangerous.

VentureBeat: Starting where we are, though, do you see anything interesting along those lines that’s coming along?

Hurlston: With the AR/VR glasses, that’s been an interesting intersection of our technology. We have these display drivers that create the ultra-HD images you can see. There’s touch that goes with it, typically, and a lot of the systems have a video processor that feeds the images into the glass. All of those things, we supply them. The AR/VR market has been good for us. It’s obviously still pretty small, but I’m much more optimistic that it’s going to take off. It plays nicely to the bag of technologies we have in the company.

Haptics is advancing. We don’t have haptics today. We do all the touch controllers on glass surfaces. Where we are trying to invest is touch on non-glass surfaces. We can see things coming — headsets are a good example, where you’re trying to touch a piece of plastic and generate sensation through there. In automobiles, on steering wheels, on things like that. We’re trying to move our touch sensors from a typical glass application to other areas where glass isn’t present, and trying to generate accuracy and precision through plastics or other materials.

VentureBeat: It’s interesting that you’re moving into IoT, and IoT devices are getting to the point where you can put AI into them. That feels like quite an advance in computing.

Hurlston: What’s going on for us, and this is something probably in your sweet spot to think about — a lot of companies now do these audio wake words, where you’re waking up a Google Home or Alexa using voice, and some simple commands are processed on the edge. The wake up doesn’t have to go to the cloud. What we’re trying to advance is a visual wake word, where we can have AI in a low-power sensor that can detect an incident, whether it’s people coming in a room or chickens moving in a coop.

We have agricultural applications for the idea, where you’re counting or sensing livestock. Counting people might apply to, do I need to turn an air conditioner on or off? Do I need to turn a display on or off? Do I need to reduce the number of people? Maybe now, in the COVID environment, you have too many people in a room. You have this low-power battery sensor that can be stuck anywhere, but rather than using voice, have a camera attached to it, a simple camera, and do some intelligence at the edge where we can identify a person or something else. Maybe the wind blowing and creating an event in front of the camera. We have a bit of inferencing and training that can happen on the device to enable those applications.

VentureBeat: It feels like we need some choices among those sensors, too. There’s a lot of places where you don’t want to put cameras, but you want that 3D detection of people or objects. You don’t want to put face recognition in a bathroom.

Hurlston: Right. That’s why these little low-power sensors can do that. They can detect motion where you don’t want to have full recognition. It can just detect that something in here is moving, so let’s turn on the lights. Particularly for industrial applications where you want to save power. It all makes sense and flows. We can have pretty high precision, where you do face recognition because there’s an AI network on the chip, but you can also just do simple motion and on/off. It just depends on how precise you need your sensor to be.

Above: Synaptics is driving into the automotive chip market.

Image Credit: Synaptics

VentureBeat: Do we credit Moore’s Law for some of this advance, being able to put more computing power into small devices? I suppose we can also credit neural networks actually working now.

Hurlston: It’s more the latter. We got reasonably good at neural networks on a high-power chip, and we were able to train the classic things. You talked about facial recognition or seeing in the dark, where we can pull out an image and train, train, train with very low light. Light turns out to be measured in luxes, which is candlelight, and we can pull out an image now with 1/16 of a lux. That’s almost total darkness. You can’t see it with your eyes, but you can pull out and enhance an image in low light.

We did that first. We developed the neural networks on high-power chips, and then migrated it to lower-power, and obviously shrunk it in the process. We were able to condense the inferencing and some of the training sequences on that low-power chip. Now we think we can deliver — it’s not going to be the same use case, but we can deliver at least an AI algorithm on a battery-powered IC.

VentureBeat: It feels like that’s important for the further existence of the planet, with things like too much cloud computing. AI at the edge is a more ecologically sound solution.

Hurlston: We’re seeing two key applications. One is obvious, and that’s power consumption. All this traffic that’s cluttering up the datacenters is consuming gigawatts, as Doc Brown would say, of power. The other one is privacy. If the decisions are made on the edge, there’s less chance that your data gets hacked and things like that. Those are the two things that people understand very simply. The third bullet is latency, making decisions much faster at the edge than having to go back to the cloud, do the calculation, and come back. But the two most important are power and privacy.

VentureBeat: Did you already have a lot of people who can do this in the company or did you have to hire a new kind of engineer to make AI and machine learning happen?

Hurlston: It’s a confluence of three things. We initially had this for video. If you look back at when we adopted it on higher-power chips that are more generally understood for machine learning, there we had to bring in our own talent. Our second step was to take an audio solution. The original idea was the wake word, following the market trend to do compute at the edge for voice. We had taken these AI and machine learning engineers, shrunk the neural network, put it into an audio chip, but we found we were behind. A lot of people can do all that wake word training. The third leg of the stool was we recently announced a partnership with a company called ETA Compute. It’s a small startup in southern California. They had a lot of machine learning and AI experts. The big language is TensorFlow, and they have the compiler that can take the TensorFlow engine and compile it into our audio chip.

The confluence of those things created this low-power AI at the edge solution that we think is different. It has quite a bit of market traction. But it’s a totally different approach to apply what I call “visual wake word” to this whole space.

VentureBeat: It seems like a good example of how AI is changing companies and industries. You wouldn’t necessarily expect it in sensing, but it makes sense that you’d have to invest in this.

Hurlston: You’ve covered technology for long enough, and you’ve been through all the cycles. Right now, the AI cycle is there. Everybody has to talk about it as part of the technology portfolio. We’re no different. We got lucky to a certain extent because we’d invested in it for a pretty clear problem, but we were able to apply it to this new situation. We have some runway.

Above: Synaptics provides chips for DisplayLink docking stations.

Image Credit: Synaptics

VentureBeat: When it comes to making these things better, either better at giving you the right information or better at the sensing, it feels like where we are with the current devices, we still need a lot of improvement. Do you see that improvement coming?

Hurlston: It comes from training data. You know better than most that it’s all about being able to provide these neural networks with the right training data. The hardest problem you have is generating datasets on which to train. Before I came here, I was at a software AI company. I spent a lot of time — we participated in a very interesting competition. The University of North Carolina had all the software AI companies together, and we were shown different dogs, from a chihuahua to a German shepherd to a pit bull. Who could best identify a dog and call it a dog from a series of pictures? They tried to throw giraffes in and things like that.

In the competition, we didn’t win, but the winner was able to get dogs to about 99% accuracy. It was amazing how well they were able to get their dataset and training to be able to identify dogs. They took the picture and they flipped it upside down, though, and nobody could get it. Once it was upside down, nobody could identify it as a dog as well as people had done when it was right side up. This thing is all about being able to train, to train on the corner cases.

This low light thing we’ve done on our video processor, we take snapshots over and over again in super low light conditions to be able to train the engine to recognize a new situation. That’s what this is all about. You know the existing situation. It’s being able to apply the existing to the new. That’s a lot harder than it sounds.

VentureBeat: If we get to the actual business, what’s doing well right now, and what do you think is going to be the source of major products in the future?

Hurlston: We’re sort of IoT of IoT. Within IoT, what our business we call IoT — we have lots of different technologies. We touched on our audio technology. That’s done very well. You have headsets that are going into a lot of work-from-home situations, with the over-ear design and active noise canceling. That business has done super well for us. We have Wi-Fi assets. We did a deal last year where we bought Broadcom’s Wi-Fi technology that they were applying to markets other than mobile phones. That business has done super well. We have docking station solutions, video processors applied to docking stations, or video conferencing systems. That’s done well for us.

In IoT, we have lots of different moving pieces, all of which are hitting at the moment, which is understandable. Work from home is good for our business. Wi-Fi in general — everything needs to be connected, and that’s driven our business. It’s been a lot of different moving parts, all of them moving simultaneously in a positive direction right now.

VentureBeat: How much emphasis do you see on IoT versus the traditional smartphone space or tablets?

Above: Synaptics is brining AI to IoT devices.

Image Credit: Synaptics

Hurlston: Smartphones is an area where we’ve done well historically as a company. Our business there was display drivers, and then the touch circuit that drives the panel. We’ll continue to play there. We’re going to approach that business, I would say, opportunistically, when we see a good opportunity to apply our technology to mobile.

But touch and display drivers — you touched on this with one of your first questions. That’s becoming more IoT-ish. Our technology that had done well in mobile, we’ll obviously continue to play in mobile where we can, but that market is competitive. A lot of players in it. Margins are tight. But what’s interesting is the market is much more open in AR/VR glasses, in games, in automobiles. We can take that same touch and display driver technology, reapply it to different end markets, and then you have something that looks more IoT-ish and commands better prices, better gross margins, things like that.

VentureBeat: As far as the role of a fabless semiconductor chip designer versus making larger systems or sub-systems, has anything changed on that front for you?

Hurlston: We’re almost entirely chips, and obviously I think that gets us further upstream of technology, given the fact that we have to drive our chips. That goes into sub-systems that ultimately go into end products. Given the lead times, we see these technical trends before others do, like this concept of the visual wake word. That’s something we’re getting out in front of.

We do sub-systems here and there. We’re unique in that context. Our historic business is the touch controllers for PCs and fingerprint sensors. Some of the PCs have fingerprint sensing for biometrics. In some cases, we’ll make that whole sub-assembly — not just the IC that does the discrimination of where your finger is, but the entire pad itself and the paint and so on. Same with the fingerprint sensor. But that’s an increasingly small part of our business. Even our historic PC business, we’re getting more into chip sales than we are into sub-assembly sales.

VentureBeat: How many people are at the company now?

Hurlston: We have about 1,400 people, most of whom are engineers, as you’d expect.

VentureBeat: On the gaming side, do you see much changing as far as the kind of detection or sensing that’s going on?

Hurlston: AR/VR is going to be a much bigger thing. For the displays, that seems to be changing a lot as well, particularly in handheld games. You have the move from some of the pioneers to go to OLED. OLED has characteristics relative to latency and other things that are not particularly ideal. You can see it move — a lot of the gaming guys are talking about mini-LED or micro-OLED, which has much faster properties than the traditional OLED. We see display changes on the horizon. We’re trying to gear our technology up for that if and when those come up.

VentureBeat: What sort of applications are you looking forward to that don’t exist today?

Hurlston: We talked about embedded touch. We talked about the push for augmented reality, although of course that’s already here. We talked about these low-power visual sensors. That’s an area in which we’re pushing. We continue to evolve our video display technology into higher resolution, both panels and displays. Obviously being able to take lower bitstreams and upconvert those — that’s where we apply a lot of our AI in the video sector, upconversion from a lower pixel count to a higher pixel count. Those are the big vectors.

With these low-power sensors, again, it comes back to getting at — in my view the big application is just solving energy. It’s not necessarily a consumer problem. But it’s not just the energy required on chip to go back and forth to the datacenter. It’s now having a lot more control of light and power and air conditioning to turn that on and off. We’re trying to take the technology, in a micro sense — it’s more environmental, and that’s obvious when you have AI at the edge. But we’re then applying it to a more macro problem, which is the useless energy consumption that happens all the time. We’re trying to drive that message and apply the technology to that problem to the extent that we can.

Above: Synaptics still makes voice biometic chips.

Image Credit: Synaptics

VentureBeat: It feels like without some of these things, IoT was either incomplete or impractical. If you didn’t have energy efficiency or AI, you were brute-forcing these things into the world. You’d either need a lot more sensors or you were causing more pollution, whether on the network or in terms of the number of devices. When you add AI and energy efficiency, it feels more sensible to deploy all these things.

Hurlston: That’s absolutely true. Maybe taking it one step further back, having wireless connectivity has been a huge enabler for these kinds of gadgets. I never imagined that I’d have a doorbell that had electronic gadgets in it. I never imagined that you’d have a bike that has electronic gadgets in it. IoT started with low-power wireless connectivity that enabled things like scales or smoke detectors or bicycles to connect to other things. That was one.

Then, to your point, the next step in the evolution has been adding AI and other sensors to a connected device to make it more useful. I’ve been surprised by how many things we’re getting into on the wireless that have this connectivity. It’s crazy stuff that you wouldn’t imagine. That was the first enabler, the low-power wireless, whether it’s Bluetooth or wireless LAN or in some instances GPS. That capability is key. We have a Bluetooth and GPS chip inside a golf ball. It’s pretty obvious what the use case is. But think about that. OK, I can find my ball when it’s at the bottom of the lake. It started with the wireless connectivity.

VentureBeat: I wrote a story about one of the companies that are doing neural networks inside hearing aids. I never thought it would be useful in that context, but apparently they’re using it to suppress noise. It recognizes the sounds you don’t want to hear and suppresses them so you only hear people talking to you.

Hurlston: Right, you have to pick out the right frequencies. Going back to your point, the second leg of the stool is certainly AI now. Whether it’s voice or visuals as we’ve been discussing, you need AI as the second leg. You’d be surprised at where you can put these simple neural networks that make a difference.

VentureBeat: The new multimedia processor you just announced, can you talk about that?

Hurlston: That’s really slotted for these set-top box applications. It was the starting point — when we talked about the AI journey, we have bigger video processors where we can do training on the chip around object detection. The big use case in this particular area is around enhancing the video, being able to upscale from a low bitrate to a higher bitrate if your feed is relatively modest, like on these Roku streamers. You can get a really low bandwidth if you’re challenged as far as your internet connection. We can upscale the video using these processors, which is what the neural network is for.

The real catalyst for us is to get into a rather bland market, which is the service provider set-top box market, where we think we have some unique advantages. We can make a good business out of that. Another cool application we just announced is a voice biometrics partnership with a company that does voice prints. Instead of just recognizing a word, you recognize the speaker. That’s running on that same processor.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform
  • networking features, and more

Become a member

Source: https://venturebeat.com/2021/02/13/michael-hurlston-how-synaptics-pivoted-from-mobile-pc-sensors-to-the-internet-of-things/

AI

Using container images to run TensorFlow models in AWS Lambda

Avatar

Published

on

TensorFlow is an open-source machine learning (ML) library widely used to develop neural networks and ML models. Those models are usually trained on multiple GPU instances to speed up training, resulting in expensive training time and model sizes up to a few gigabytes. After they’re trained, these models are deployed in production to produce inferences. They can be synchronous, asynchronous, or batch-based workloads. Those endpoints need to be highly scalable and resilient in order to process from zero to millions of requests. This is where AWS Lambda can be a compelling compute service for scalable, cost-effective, and reliable synchronous and asynchronous ML inferencing. Lambda offers benefits such as automatic scaling, reduced operational overhead, and pay-per-inference billing.

This post shows you how to use any TensorFlow model with Lambda for scalable inferences in production with up to 10 GB of memory. This allows us to use ML models in Lambda functions up to a few gigabytes. For this post, we use TensorFlow-Keras pre-trained ResNet50 for image classification.

Overview of solution

Lambda is a serverless compute service that lets you run code without provisioning or managing servers. Lambda automatically scales your application by running code in response to every event, allowing event-driven architectures and solutions. The code runs in parallel and processes each event individually, scaling with the size of the workload, from a few requests per day to hundreds of thousands of workloads. The following diagram illustrates the architecture of our solution.

You can package your code and dependencies as a container image using tools such as the Docker CLI. The maximum container size is 10 GB. After the model for inference is Dockerized, you can upload the image to Amazon Elastic Container Registry (Amazon ECR). You can then create the Lambda function from the container imaged stored in Amazon ECR.

Prerequisites

For this walkthrough, you should have the following prerequisites:

Implementing the solution

We use a pre-trained model from the TensorFlow Hub for image classification. When an image is uploaded to an Amazon Simple Storage Service (Amazon S3) bucket, a Lambda function is invoked to detect the image and print it to the Amazon CloudWatch logs. The following diagram illustrates this workflow.

To implement the solution, complete the following steps:

  1. On your local machine, create a folder with the name lambda-tensorflow-example.
  2. Create a requirements.txt file in that directory.
  3. Add all the needed libraries for your ML model. For this post, we use TensorFlow 2.4.
  4. Create an app.py script that contains the code for the Lambda function.
  5. Create a Dockerfile in the same directory.

The following text is an example of the requirements.txt file to run TensorFlow code for our use case:

# List all python libraries for the lambda
tensorflow==2.4.0
tensorflow_hub==0.11
Pillow==8.0.1

We’re using the TensorFlow 2.4 version with CPU support only because, as of this writing, Lambda only offers CPU support. For more information about CPU-only versions of TensorFlow, see Package location.

The Python code is placed in app.py. The inference function in app.py needs to follow a specific structure to be invoked by the Lambda runtime. For more information about handlers for Lambda, see AWS Lambda function handler in Python. See the following code:

import json
import boto3
import numpy as np
import PIL.Image as Image import tensorflow as tf
import tensorflow_hub as hub IMAGE_WIDTH = 224
IMAGE_HEIGHT = 224 IMAGE_SHAPE = (IMAGE_WIDTH, IMAGE_HEIGHT)
model = tf.keras.Sequential([hub.KerasLayer("model/")])
model.build([None, IMAGE_WIDTH, IMAGE_HEIGHT, 3]) imagenet_labels= np.array(open('model/ImageNetLabels.txt').read().splitlines())
s3 = boto3.resource('s3') def lambda_handler(event, context): bucket_name = event['Records'][0]['s3']['bucket']['name'] key = event['Records'][0]['s3']['object']['key'] img = readImageFromBucket(key, bucket_name).resize(IMAGE_SHAPE) img = np.array(img)/255.0 prediction = model.predict(img[np.newaxis, ...]) predicted_class = imagenet_labels[np.argmax(prediction[0], axis=-1)] print('ImageName: {0}, Prediction: {1}'.format(key, predicted_class)) def readImageFromBucket(key, bucket_name): bucket = s3.Bucket(bucket_name) object = bucket.Object(key) response = object.get() return Image.open(response['Body'])

The following Dockerfile for Python 3.8 uses the AWS provided open-source base images that can be used to create container images. The base images are preloaded with language runtimes and other components required to run a container image on Lambda.

# Pull the base image with python 3.8 as a runtime for your Lambda
FROM public.ecr.aws/lambda/python:3.8 # Install tar and gzip
RUN yum -y install tar gzip zlib # Copy the earlier created requirements.txt file to the container
COPY requirements.txt ./ # Install the python requirements from requirements.txt
RUN python3.8 -m pip install -r requirements.txt # Copy the earlier created app.py file to the container
COPY app.py ./ # Download ResNet50 and store it in a directory
RUN mkdir model
RUN curl -L https://tfhub.dev/google/imagenet/resnet_v1_50/classification/4?tf-hub-format=compressed -o ./model/resnet.tar.gz
RUN tar -xf model/resnet.tar.gz -C model/
RUN rm -r model/resnet.tar.gz # Download ImageNet labels
RUN curl https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt -o ./model/ImageNetLabels.txt # Set the CMD to your handler
CMD ["app.lambda_handler"]

Your folder structure should look like the following screenshot.

You can build and push the container image to Amazon ECR with the following bash commands. Replace the <AWS_ACCOUNT_ID> with your own AWS account ID and also specify a <REGION>.

# Build the docker image
docker build -t lambda-tensorflow-example . # Create a ECR repository
aws ecr create-repository --repository-name lambda-tensorflow-example --image-scanning-configuration scanOnPush=true --region <REGION> # Tag the image to match the repository name
docker tag lambda-tensorflow-example:latest <AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com/lambda-tensorflow-example:latest # Register docker to ECR
aws ecr get-login-password --region <REGION> | docker login --username AWS --password-stdin <AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com # Push the image to ECR
docker push <AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com/lambda-tensorflow-example:latest

If you want to test your model inference locally, the base images for Lambda include a Runtime Interface Emulator (RIE) that allows you to also locally test your Lambda function packaged as a container image to speed up the development cycles.

Creating an S3 bucket

As a next step, we create an S3 bucket to store the images used to predict the image class.

  1. On the Amazon S3 console, choose Create bucket.
  2. Give the S3 bucket a name, such as tensorflow-images-for-inference-<Random_String> and replace the <Random_String> with a random value.
  3. Choose Create bucket.

Creating the Lambda function with the TensorFlow code

To create your Lambda function, complete the following steps:

  1. On the Lambda console, choose Functions.
  2. Choose Create function.
  3. Select Container image.
  4. For Function name, enter a name, such as tensorflow-endpoint.
  5. For Container image URI, enter the earlier created lambda-tensorflow-example repository.

  1. Choose Browse images to choose the latest image.
  2. Click Create function to initialize the creation of it.
  3. To improve the Lambda runtime, increase the function memory to at least 6 GB and timeout to 5 minutes in the Basic settings.

For more information about function memory and timeout settings, see New for AWS Lambda – Functions with Up to 10 GB of Memory and 6 vCPUs.

Connecting the S3 bucket to your Lambda function

After the successful creation of the Lambda function, we need to add a trigger to it so that whenever a file is uploaded to the S3 bucket, the function is invoked.

  1. On the Lambda console, choose your function.
  2. Choose Add trigger.

  1. Choose S3.
  2. For Bucket, choose the bucket you created earlier.

After the trigger is added, you need to allow the Lambda function to connect to the S3 bucket by setting the appropriate AWS Identity and Access Management (IAM) rights for its execution role.

  1. On the Permissions tab for your function, choose the IAM role.
  2. Choose Attach policies.
  3. Search for AmazonS3ReadOnlyAccess and attach it to the IAM role.

Now you have configured all the necessary services to test your function. Upload a JPG image to the created S3 bucket by opening the bucket in the AWS management console and clicking Upload. After a few seconds, you can see the result of the prediction in the CloudWatch logs. As a follow-up step, you could store the predictions in an Amazon DynamoDB table.

After uploading a JPG picture to the S3 bucket we will get the predicted image class as a result printed to CloudWatch. The Lambda function will be triggered by EventBridge and pull the image from the bucket. As an example, we are going to use the picture of this parrot to get predicted by our inference endpoint.

In the CloudWatch logs the predicted class is printed. Indeed, the model predicts the correct class for the picture (macaw):

Performance

In order to achieve optimal performance, you can try various levels of memory setting (which linearly changes the assigned vCPU, to learn more, read this AWS News Blog). In the case of our deployed model, we realize most performance gains at about 3GB – 4GB (~2vCPUs) setting and gains beyond that are relatively low. Different models see different level of performance improvement by increased amount of CPU so it is best to determine this experimentally for your own model. Additionally, it is highly recommended that you compile your source code to take advantage of Advanced Vector Extensions 2 (AVX2) on Lambda that further increases the performance by allowing vCPUs to run higher number of integer and floating-point operations per clock cycle.

Conclusion

Container image support for Lambda allows you to customize your function even more, opening up a lot of new use cases for serverless ML. You can bring your custom models and deploy them on Lambda using up to 10 GB for the container image size. For smaller models that don’t need much computing power, you can perform online training and inference purely in Lambda. When the model size increases, cold start issues become more and more important and need to be mitigated. There is also no restriction on the framework or language with container images; other ML frameworks such as PyTorch, Apache MXNet, XGBoost, or Scikit-learn can be used as well!

If you do require GPU for your inference, you can consider using containers services such as Amazon Elastic Container Service (Amazon ECS), Kubernetes, or deploy the model to an Amazon SageMaker endpoint.


About the Author

Jan Bauer is a Cloud Application Developer at AWS Professional Services. His interests are serverless computing, machine learning, and everything that involves cloud computing.

Source: https://aws.amazon.com/blogs/machine-learning/using-container-images-to-run-tensorflow-models-in-aws-lambda/

Continue Reading

AI

IBM Reportedly Retreating from Healthcare with Watson 

Avatar

Published

on

IBM is reported to be considering a sale of Watson Health, an indication of the challenges of applying AI in healthcare. IBM continues to invest in cloud services using Watson. (Photo by Carson Masterson on Unsplash.)  

By John P. Desmond, AI Trends Editor  

Reports surfaced last week that IBM is contemplating a sale of Watson Health, representing a retreat from the market of AI applied to healthcare that IBM had pursued under the direction of its previous CEO. 

The Wall Street Journal last week reported IBM was exploring the sale of Watson Health; IBM did not confirm the report. Ten years ago, when IBM Watson won on the Jeopardy! game show against two of the game’s record winners, the Watson brand in AI was established. 

As reported in AI Trends last February, the day after Watson defeated the two human champions on Jeopardy!, IBM announced Watson was heading into the medical field. IBM would take its ability to understand natural language that it showed off on television, and apply it to medicine. The first commercial offerings would be available in 18 to 24 months, the company promised, according to an account in IEEE Spectrum from April 2019.  

It was a tough road. IBM was the first company to make a major push to bring AI to medicine. The alarm was sounded by Robert Wachter, chair of the department of medicine at the University of California, San Francisco, and author of the 2015 book The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age (McGraw-Hill). The Watson win on Jeopardy! Gave the IBM AI salesforce a launching pad.  

“They came in with marketing first, product second, and got everybody excited,” stated Wachter. “Then the rubber hit the road. This is an incredibly hard set of problems, and IBM, by being first out, has demonstrated that for everyone else.”  

Then-IBM CEO Ginni Rometty Used Watson Victory to Launch AI in Healthcare  

Ginni Rometty, IBM’s CEO at the time, told an audience of health IT professionals at a 2017 conference that “AI is mainstream, it’s here, and it can change almost everything about health care.” She, like many, saw the potential for AI to help transform the healthcare industry. 

Watson had used advances in natural language processing to win at Jeopardy. The Watson team used machine learning on a training dataset of Jeopardy clues and responses. To enter the healthcare market, IBM tried using text recognition on medical records to build its knowledge base. Unstructured data such as doctors’ notes full of jargon and shorthand may account for 80% of a patient’s record. It was challenging.   

The effort was to build a diagnostic tool. IBM formed the Watson Health division in 2015. The unit made $4 billion of acquisitions. The search continued for the medical business case to justify the investments. Many projects were launched around decision support using large medical data sets. A focus on oncology to personalize cancer treatment for patients looked promising.  

Physicians at the University of Texas MD Anderson Cancer Center in Houston, worked with IBM to create a tool called Oncology Expert Advisor. MD Anderson got the tool to test stage in the leukemia department; it never became a commercial product.   

The project did not end well; it was cancelled in 2016. An audit by the University of Texas found the cancer center had spent $62 million on the project. The IEEE Spectrum authors said the project revealed “a fundamental mismatch between the promise of machine learning and the reality of medical care,” something that would be useful to today’s doctors.  

IBM made a round of layoffs in the IBM Watson Health unit in 2018, according to another report at the time by IEEE Spectrum in June 2018. Engineers from one of the companies IBM had acquired, Phytel, reported a shrinking client base for its patient analytics solution from 150 to 80 since the acquisition. “Smaller companies are eating us alive,” stated the engineer. “They’re better, faster, cheaper. They’re winning our contracts, taking our customers, doing better at AI.”  

Mismatch Seen Between Realities of Healthcare and Promise of AI  

Dr. Thomas J. Fuchs, Dean of AI and Human Health, Mount Sinai Health System

This notion of a mismatch between the promise of AI and realities of healthcare was seconded in last week’s Wall Street Journal report that tech companies may lack the deep expertise in how healthcare works in patient settings. “You truly have to understand the clinical workflow in the trenches,” stated Thomas J. Fuchs, Mount Sinai Health System’s dean of artificial intelligence and human health. “You have to understand where you can insert AI and where it can be helpful” without slowing things down in the clinic. 

Packaging AI advances in computer science into a viable software product or service has always been a fundamental challenge in the software business. “Watson may be very emblematic of a broader issue at IBM of taking good science and finding a way to make it commercially relevant,” stated Toni Sacconaghi, an analyst at Bernstein Research.  

Toni Sacconaghi, analyst, Bernstein Research

New IBM CEO Arvind Krishna has said AI along with hybrid cloud computing, would be pivotal for IBM going forward. (See AI Trends, November 2020.) Krishna is moving to exit struggling business units and concentrate on those that can deliver consistent growth. As part of this effort, IBM is in the process of spinning its managed IT services division out into a new public company; IT services is seen as a declining margin business by analysts. IBM had $100 billion in sales in 2010 and $73.6 billion last year. 

Another challenge for AI in healthcare is the lack of data-collection standards, which makes applying models developed in one healthcare setting and applying it in others is difficult. “The customization problem is severe in healthcare,” stated Andrew Ng, an AI expert and CEO of startup Landing AI, based in Palo Alto, Calif., to The Wall Street Journal. 

Healthcare markets where AI has shown promise and achieved results include radiology and pathology, where image recognition techniques can be used to answer specific questions. Also, AI has made inroads in streamlining business processes such as billing and charting, which can help save money and free up staff to focus on more challenging areas. Administrative costs are said to be 30 percent of healthcare costs. 

Meanwhile, investment for AI in healthcare continues, with spending projected to grow at an annualized rate of 48% through 2023, according to a recent report from Business Insider. New players include giants such as Google, which has defined a Cloud Healthcare application programming interface (API), that can take data from users’ electronic health records via machine learning, with the aim of helping physicians make more informed clinical decisions. Google is also working with the University of California, Stanford University, and the University of Chicago on an AI system to predict the outcomes of hospital visits 

AI is also being applied to the move to personalized healthcare, for example with wearable technology such as FitBits and smartwatches, which can alert users and healthcare professionals to potential health issues and risks.  

While retreating from applying Watson in healthcare, IBM is expanding the role of Watson in its cloud service offerings. These include natural language processing, sentiment analysis and virtual assistants, according to entries on the IBM Watson blog,  

Read the source articles and information in The Wall Street Journalin IEEE Spectrum from April 2019, in AI Trends February 2020, in IEEE Spectrum from June 2018, AI Trends, November 2020, from Business Insider and on the IBM Watson blog.  

Source: https://www.aitrends.com/healthcare/ibm-reportedly-retreating-from-healthcare-with-watson/

Continue Reading

AI

SolarWinds Hackers Targeted Cloud Services as a Key Objective 

Avatar

Published

on

The SolarWinds attackers appear to have as a primary objective the compromise the authentication method for cloud services, with far-reaching implications. (Credit: Getty Images).   

By John P. Desmond, AI Trends Editor 

The SolarWinds hackers appeared to have targeted cloud services as a key objective, potentially giving them access to many, if not all, of an organization’s cloud-based services.  

Christopher Budd, independent security expert

This is from an account in GeekWire written by Christopher Budd, an independent security consultant who worked previously in Microsoft’s Security Response Center for 10 years.  

“If we decode the various reports and connect the dots we can see that the SolarWinds attackers have targeted authentication systems on the compromised networks, so they can log in to cloud-based services like Microsoft Office 365 without raising alarms,” wrote Budd. “Worse, the way they’re carrying this out can potentially be used to gain access to many, if not all, of an organization’s cloud-based services.”  

The implication is that those assessing the impact of the attacks need to look not just at their own systems and networks, but also at their cloud-based services for evidence of compromise. And it means that defending against attacks means increasing the security and monitoring of cloud services authentication systems, “from now on.”  

Budd cited these key takeaways: 

  • After establishing a foothold in a network, the SolarWinds attackers target the systems that issue proof of identity used by cloud-based services; and they steal the means used to issue IDs; 
  • Once they have this ability, they are able to create fake IDs that allow them to impersonate legitimate users, or create malicious accounts that seem legitimate, including accounts with administrative access;  
  • Because the IDs are used to provide access to data and service by cloud-based accounts, the attackers are able to access data and email as if they were legitimate users.

SAML Authentication Method for Cloud Services Seen Targeted 

Cloud-based services use an authentication method called Security Assertion Markup Language (SAML), which issues a token that is “proof” of the identity of a legitimate user to the services. Budd ascertained, based on a series of posts on the Microsoft blog, that the SAML service was targeted. While this type of attack was first seen in 2017, “This is the first major attack with this kind of broad visibility that targets cloud-based authentication mechanisms,” Budd stated. 

In response to a question Budd asked Microsoft, on whether the company learned of any vulnerabilities that led to this attack, he got this response: “We have not identified any Microsoft product or cloud service vulnerabilities in these investigations. Once in a network, the intruder then uses the foothold to gain privilege and use that privilege to gain access.” 

A response from the National Security Administration was similar, saying the attackers, by “abusing the federated authentication,” were not exploiting any vulnerability in the Microsoft authentication system, “but rather abusing the trust established across the integrated components.” 

Also, although the SolarWinds attack came through a Microsoft cloud-based service, it involved the SAML open standard that is widely used by vendors of cloud-based services, not just Microsoft. “The SolarWinds attacks and these kinds of SAML-based attacks against cloud services in the future can involve non-Microsoft SAML-providers and cloud service providers,” Budd stated. 

American Intelligence Sees Attack Originating with Russia’s Cozy Bear 

American intelligence officials believe the attack originated from Russia. Specifically, according to a report from The Economist, the group of attackers known as Cozy Bear, thought to be part of Russia’s intelligence service, were responsible. “It appears to be one of the largest-ever acts of digital espionage against America,” the account stated.  

The attack demonstrated “top-tier operational tradecraft,” according to FireEye, a cyber-security firm that also was itself a victim.  

America has tended to categorize and respond to cyber-attacks happening over the last decade according to the aims of the attackers. It has regarded intrusions intended to steal secretsold-fashioned espionageas fair game that the US National Security Agency is also engaged in. But attacks intended to cause harm, such as the North Korea assault on Sony Pictures in 2014, or China’s theft of industrial secrets, are viewed as crossing a line, the account suggested. Thus, sanctions have been imposed on many Russian, Chinese, North Korean and Iranian hackers.   

The Solar Winds attack seems to have created its own category. “This effort to stamp norms onto a covert and chaotic arena of competition has been unsuccessful,” the Economist account stated. “The line between espionage and subversion is blurred.”  

One observer sees that America has grown less tolerant of “what’s allowed in cyberspace” since the hack of the Officer of Personnel Management (OPM) in 2015. That hack breached OPM networks and exposed the records of 22.1 million related to government employees, others who had undergone background checks, and friends and family. State-sponsored hackers working on behalf of the Chinese government were believed responsible.   

“Such large-scale espionage “would be now at the top of the list of operations that they would deem as unacceptable,” stated Max Smeets of the Centre of Security Studies in Zurich. 

“On-Prem” Software Seen as More Risky 

The SolarWinds Orion product is installed “on-prem,” meaning it is installed and run on computers on the premises of the organization using the software. Such products carry security risks that IT leadership needs to carefully evaluate, suggested a recent account in eWeek 

William White, security and IT director, BigPanda

The SolarWinds attackers apparently used a compromised software patch to gain entry, suggested William White, security and IT director of BigPanda, which offers AI software to detect and analyze problems in IT systems. “With on-prem software, you often have to grant elevated permissions or highly privileged accounts for the software to run, which creates risk,” he stated.    

Because the SolarWinds attack was apparently executed through a software patch, “Ironically, the most exposed SolarWinds customers were the ones that were actually diligent about installing Orion patches,” stated White.  

Read the source articles in GeekWirefrom The Economist and in eWeek.

Source: https://www.aitrends.com/security/solarwinds-hackers-targeted-cloud-services-as-a-key-objective/

Continue Reading

AI

RAND Corp. Finds DoD “Significantly Challenged” in AI Posture 

Avatar

Published

on

A new report from RAND Corp. finds the US DoD’s AI posture faces challenges around data and testing that ensures performance and safety. (Credit: Getty Images)  

By AI Trends Staff  

In a recently-released updated evaluation of the posture of the US Department of Defense (DoD) on artificial intelligence, researchers at RAND Corpfound that “despite some positive signs, the DoD’s posture is significantly challenged across all dimensions” of the assessment. 

The RAND researchers were asked by Congress, within the 2019 National Defense Authorization Act (NDAA), and the director of DoD’s Joint Artificial Intelligence Center (JAIC), to help answer the question: “Is DoD ready to leverage AI technologies and take advantage of the potential associated with them, or does it need to take major steps to position itself to use those technologies effectively and safely and scale up their use?” 

The term artificial intelligence was first coined in 1956 at a conference at Dartmouth College that showcased a program designed to mimic human thinking skills. Almost immediately thereafter, the Defense Advanced Research Projects Agency (DARPA) (then known as the Advanced Research Projects Agency [ARPA]), the research arm of the military, initiated several lines of research aimed at applying AI principles to defense challenges.   

Danielle Tarraf, Senior Information Scientist, RAND Corp.

Since the 1950s, AI—and its subdiscipline of machine learning (ML)—has come to mean many different things to different people, stated the report, whose lead author is Danielle C. Tarraf, a senior information scientist at RAND and a professor at the RAND Graduate School. (RAND Corp. is a US nonprofit think tank created in 1948 to offer research and analysis to the US Armed Forces.)    

For example, the 2019 NDAA cited as many as five definitions of AI. “No consensus emerged on a common definition from the dozens of interviews conducted by the RAND team for its report to Congress,” the RAND report stated.  

The RAND researchers decided to remain flexible and not be bound by precise definitions. Instead, they tried to answer the question of whether the DoD is positioned to build or acquire, test, transition and sustain—at scale—a set of technologies broadly falling under the AI umbrella? And if not, what would DoD need to do to get there? Considering the implications of AI for DoD strategic decision makers, the researchers concentrated on three elements and how they interact:  

  • the technology and capabilities space 
  • the spectrum of DoD AI applications 
  • the investment space and time horizon.

While algorithms underpin most AI solutions, interest and hype is fueled by advances in AI, such as deep learning. This requires large data sets, and which tend to be highly-specific to the applications for which they were designed, most of which are commercial. Referring to AI verification, validation, test and evaluation (VVT&E) procedures critical to the function of software in the DoD,  the researchers stated, “VVT&E remains very challenging across the board for all AI applications, including safety-critical military applications.”  

The researchers divided AI applications for DoD into three groups:  

  • Enterprise AI, including applications such as the management of health records at military hospitals in well-controlled environments;  
  • Mission-Support AI, including applications such as the Algorithmic Warfare Cross-Functional Team (also known as Project Maven), which aims to use machine learning to assist humans in analyzing large volumes of imagery from video data collected in the battle theater by drones, and;  
  • Operational AI, including applications of AI integrated into weapon systems that must contend with dynamic, adversarial environments, and that have significant implications in the case of failure for casualties. 

Realistic goals need to be set for how long AI will need to progress from demonstrations of what is possible to full-scale implementations in the field. The RAND team’s analysis suggests at-scale deployments in the:   

  • near term (up to five years) for enterprise AI 
  • middle term (five to ten years) for most mission-support AI, and  
  • far term (longer than ten years) for most operational AI applications. 

The RAND team sees the following challenges for AI at the DoD:  

  • Organizationally, the current DoD AI strategy lacks both baselines and metrics for assessing progress. And the JAIC has not been given the authority, resources, and visibility needed to scale AI and its impact DoD-wide. 
  • Data are often lacking, and when they exist, they often lack traceability, understandability, accessibility, and interoperability. 
  • The current state of VVT&E for AI technologies cannot ensure the performance and safety of AI systems, especially those that are safety-critical. 
  • DoD lacks clear mechanisms for growing, tracking, and cultivating AI talent, a challenge that is only going to grow with the increasingly tight competition with academia, the commercial world, and other kinds of workspaces for individuals with the needed skills and training. 
  • Communications channels among the builders and users of AI within DoD are sparse. 

The researchers made a number of recommendations to address these issues. 

Two Challenge Areas Addressed  

Two of these challenge areas have been recently addressed at a meeting hosted by the AFCEA, the professional association that links people in military, government, industry and academia, reported in an account in FCW. The organization engages in the “ethical exchange of information” and has roots in the US Civil War, according to its website.   

Jacqueline Tame is Acting Deputy Director at the JAIC, whose years of experience include positions with the House Permanent Select Committee on Intelligence, work with an AI analytics platform for the Office of the Secretary of Defense and then positions in the JAIC. She has graduate degrees from the Naval War College and the LBJ School of Public Affairs.  

She addressed how AI at DoD is running into culture and policy norms in conflict with its capability. For example, “We still have over… several thousand security classification guidance documents in the Department of Defense alone.” The result is a proliferation of “data owners.” She commented, “That is antithetical to the idea that data is a strategic asset for the department.” 

She used the example of predictive maintenance, which requires analysis of data from a range of sources to be effective, as an infrastructure challenge for the DoD currently. “This is a warfighting issue,” Tame stated. “To make AI effective for warfighting applications, we have to stop thinking about it in these limited stovepiped ways.” 

Jane Pinelis, chief of testing and evaluation, JAIC

Data standards need to be set and unified, suggested speaker Jane Pinelis, the chief of testing and evaluation for the JAIC. Her background includes time at the Johns Hopkins University Applied Physics Laboratory, where she was involved in “algorithmic warfare.” She is also a veteran of the Marine Corps, where her assignments included a position in the Warfighting Lab. She holds a PhD in Statistics from the University of Michigan. 

“Standards are elevated best practices and we don’t necessarily have best practices yet,” Pinelis stated. JAIC is working on it, by collecting and documenting best practices and leading a working group in the intelligence community on data collection and tagging. 

Weak data readiness has been an impediment to AI for the DoD, she stated. In response, the JAIC is preparing multiple award contracts for test and evaluation and data readiness, expected soon.  

Read the source articles and information from RAND Corp. and FCW. 

Source: https://www.aitrends.com/ai-in-government/rand-corp-finds-dod-significantly-challenged-in-ai-posture/

Continue Reading
SPACS5 days ago

TPB Acquisition I files for a $250 million IPO; SPAC targets sustainable food production

Amb Crypto4 days ago

Litecoin Price Analysis: 20 February

SPACS5 days ago

Elliott Management’s tech-focused SPAC Elliott Opportunity I files for a $1.0 billion IPO

Blockchain4 days ago

VeChain Review: Blockchain Supply Chain Management

PR Newswire4 days ago

S3 AeroDefense Signs 10 Year Distribution Agreement & Repair License with Honeywell Aerospace

SPACS5 days ago

Special purpose acquisition companies grow in popularity on Wall Street during the pandemic

Amb Crypto4 days ago

Why MicroStrategy and other institutions don’t regret their Bitcoin buys

Amb Crypto4 days ago

Why retail adoption of Bitcoin may be a challenge at $55,000

SPACS5 days ago

Paul Singer-backed blank-check firms file for up to $1.5 billion IPO

NEWATLAS5 days ago

Perseverance sends back post-touchdown selfie

Amb Crypto5 days ago

Chainlink integrates with Danal Fintech to support retail Bitcoin payments

Automotive4 days ago

Tesla’s Gigafactory formula rose from a humble “tent” at the Fremont Factory

Amb Crypto4 days ago

Are Bitcoin’s long-term hodlers entering the seller’s market?

Amb Crypto4 days ago

Polkadot, Cosmos, IOTA Price Analysis: 20 February

NEWATLAS5 days ago

Super-light laptop carries heavyweight price tag

Amb Crypto4 days ago

Chainlink, Aave, SushiSwap Price Analysis: 20 February

Amb Crypto5 days ago

SEC is “dead wrong” in the Ripple case claims former SEC chair Mary Jo White

Amb Crypto5 days ago

Litecoin, EOS, Decred Price Analysis: 20 February

Amb Crypto5 days ago

Ethereum Price Analysis: 20 February

Amb Crypto4 days ago

Binance coin, Tron, FTX Token Price Analysis: 20 February

Trending