Connect with us


Building a visual search application with Amazon SageMaker and Amazon ES




Sometimes it’s hard to find the right words to describe what you’re looking for. As the adage goes, “A picture is worth a thousand words.” Often, it’s easier to show a physical example or image than to try to describe an item with words, especially when using a search engine to find what you’re looking for.

In this post, you build a visual image search application from scratch in under an hour, including a full-stack web application for serving the visual search results.

Visual search can improve customer engagement in retail businesses and e-commerce, particularly for fashion and home decoration retailers. Visual search allows retailers to suggest thematically or stylistically related items to shoppers, which retailers would struggle to achieve by using a text query alone. According to Gartner, “By 2021, early adopter brands that redesign their websites to support visual and voice search will increase digital commerce revenue by 30%.”

High-level example of visual searching

Amazon SageMaker is a fully managed service that provides every developer and data scientist with the ability to build, train, and deploy machine learning (ML) models quickly. Amazon Elasticsearch Service (Amazon ES) is a fully managed service that makes it easy for you to deploy, secure, and run Elasticsearch cost-effectively at scale. Amazon ES offers k-Nearest Neighbor (KNN) search, which can enhance search in similar use cases such as product recommendations, fraud detection, and image, video, and semantic document retrieval. Built using the lightweight and efficient Non-Metric Space Library (NMSLIB), KNN enables high-scale, low-latency, nearest neighbor search on billions of documents across thousands of dimensions with the same ease as running any regular Elasticsearch query.

The following diagram illustrates the visual search architecture.

Overview of solution

Implementing the visual search architecture consists of two phases:

  1. Building a reference KNN index on Amazon ES from a sample image dataset.
  2. Submitting a new image to the Amazon SageMaker endpoint and Amazon ES to return similar images.

KNN reference index creation

In this step, from each image you extract 2,048 feature vectors from a pre-trained Resnet50 model hosted in Amazon SageMaker. Each vector is stored to a KNN index in an Amazon ES domain. For this use case, you use images from FEIDEGGER, a Zalando research dataset consisting of 8,732 high-resolution fashion images. The following screenshot illustrates the workflow for creating KNN index.

The process includes the following steps:

  1. Users interact with a Jupyter notebook on an Amazon SageMaker notebook instance.
  2. A pre-trained Resnet50 deep neural net from Keras is downloaded, the last classifier layer is removed, and the new model artifact is serialized and stored in Amazon Simple Storage Service (Amazon S3). The model is used to start a TensorFlow Serving API on an Amazon SageMaker real-time endpoint.
  3. The fashion images are pushed through the endpoint, which runs the images through the neural network to extract the image features, or embeddings.
  4. The notebook code writes the image embeddings to the KNN index in an Amazon ES domain.

Visual search from a query image

In this step, you present a query image from the application, which passes through the Amazon SageMaker hosted model to extract 2,048 features. You use these features to query the KNN index in Amazon ES. KNN for Amazon ES lets you search for points in a vector space and find the “nearest neighbors” for those points by Euclidean distance or cosine similarity (the default is Euclidean distance). When it finds the nearest neighbors vectors (for example, k = 3 nearest neighbors) for a given image, it returns the associated Amazon S3 images to the application. The following diagram illustrates the visual search full-stack application architecture.

The process includes the following steps:

  1. The end-user accesses the web application from their browser or mobile device.
  2. A user-uploaded image is sent to Amazon API Gateway and AWS Lambda as a base64 encoded string and is re-encoded as bytes in the Lambda function.
    1. A publicly readable image URL is passed as a string and downloaded as bytes in the function.
  3. The bytes are sent as the payload for inference to an Amazon SageMaker real-time endpoint, and the model returns a vector of the image embeddings.
  4. The function passes the image embedding vector in the search query to the k-nearest neighbor in the index in the Amazon ES domain. A list of k similar images and their respective Amazon S3 URIs is returned.
  5. The function generates pre-signed Amazon S3 URLs to return back to the client web application, used to display similar images in the browser.

AWS services

To build the end-to-end application, you use the following AWS services:

  • AWS AmplifyAWS Amplify is a JavaScript library for front-end and mobile developers building cloud-enabled applications. For more information, see the GitHub repo.
  • Amazon API Gateway – A fully managed service to create, publish, maintain, monitor, and secure APIs at any scale.
  • AWS CloudFormationAWS CloudFormation gives developers and businesses an easy way to create a collection of related AWS and third-party resources and provision them in an orderly and predictable fashion.
  • Amazon ES – A managed service that makes it easy to deploy, operate, and scale Elasticsearch clusters at scale.
  • AWS IAMAWS Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely.
  • AWS Lambda – An event-driven, serverless computing platform that runs code in response to events and automatically manages the computing resources the code requires.
  • Amazon SageMaker – A fully managed end-to-end ML platform to build, train, tune, and deploy ML models at scale.
  • AWS SAMAWS Serverless Application Model (AWS SAM) is an open-source framework for building serverless applications.
  • Amazon S3 – An object storage service that offers an extremely durable, highly available, and infinitely scalable data storage infrastructure at very low cost.


For this walkthrough, you should have an AWS account with appropriate IAM permissions to launch the CloudFormation template.

Deploying your solution

You use a CloudFormation stack to deploy the solution. The stack creates all the necessary resources, including the following:

  • An Amazon SageMaker notebook instance to run Python code in a Jupyter notebook
  • An IAM role associated with the notebook instance
  • An Amazon ES domain to store and retrieve image embedding vectors into a KNN index
  • Two S3 buckets: one for storing the source fashion images and another for hosting a static website

From the Jupyter notebook, you also deploy the following:

  • An Amazon SageMaker endpoint for getting image feature vectors and embeddings in real time.
  • An AWS SAM template for a serverless back end using API Gateway and Lambda.
  • A static front-end website hosted on an S3 bucket to demonstrate a real-world, end-to-end ML application. The front-end code uses ReactJS and the Amplify JavaScript library.

To get started, complete the following steps:

  1. Sign in to the AWS Management Console with your IAM user name and password.
  2. Choose Launch Stack and open it in a new tab:
  3. On the Quick create stack page, select the check box to acknowledge the creation of IAM resources.
  4. Choose Create stack.
  5. Wait for the stack to complete executing.

You can examine various events from the stack creation process on the Events tab. When the stack creation is complete, you see the status CREATE_COMPLETE.

You can look on the Resources tab to see all the resources the CloudFormation template created.

  1. On the Outputs tab, choose the SageMakerNotebookURL value.

This hyperlink opens the Jupyter notebook on your Amazon SageMaker notebook instance that you use to complete the rest of the lab.

You should be on the Jupyter notebook landing page.

  1. Choose visual-image-search.ipynb.

Building a KNN index on Amazon ES

For this step, you should be at the beginning of the notebook with the title Visual image search. Follow the steps in the notebook and run each cell in order.

You use a pre-trained Resnet50 model hosted on an Amazon SageMaker endpoint to generate the image feature vectors (embeddings). The embeddings are saved to the Amazon ES domain created in the CloudFormation stack. For more information, see the markdown cells in the notebook.

Continue when you reach the cell Deploying a full-stack visual search application in your notebook.

The notebook contains several important cells.

To load a pre-trained ResNet50 model without the final CNN classifier layer, see the following code (this model is used just as an image feature extractor):

#Import Resnet50 model
model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False,input_shape=(3, 224, 224),pooling='avg')

You save the model as a TensorFlow SavedModel format, which contains a complete TensorFlow program, including weights and computation. See the following code:

#Save the model in SavedModel format'./export/Servo/1/', save_format='tf')

Upload the model artifact (model.tar.gz) to Amazon S3 with the following code:

#Upload the model to S3
sagemaker_session = sagemaker.Session()
inputs = sagemaker_session.upload_data(path='model.tar.gz', key_prefix='model')

You deploy the model into an Amazon SageMaker TensorFlow Serving-based server using the Amazon SageMaker Python SDK. The server provides a super-set of the TensorFlow Serving REST API. See the following code:

#Deploy the model in Sagemaker Endpoint. This process will take ~10 min.
from sagemaker.tensorflow.serving import Model sagemaker_model = Model(entry_point='', model_data = 's3://' + sagemaker_session.default_bucket() + '/model/model.tar.gz', role = role, framework_version='2.1.0', source_dir='./src' ) predictor = sagemaker_model.deploy(initial_instance_count=3, instance_type='ml.m5.xlarge')

Extract the reference images features from the Amazon SageMaker endpoint with the following code:

# define a function to extract image features
from time import sleep sm_client = boto3.client('sagemaker-runtime')
ENDPOINT_NAME = predictor.endpoint def get_predictions(payload): return sm_client.invoke_endpoint(EndpointName=ENDPOINT_NAME, ContentType='application/x-image', Body=payload) def extract_features(s3_uri): key = s3_uri.replace(f's3://{bucket}/', '') payload = s3.get_object(Bucket=bucket,Key=key)['Body'].read() try: response = get_predictions(payload) except: sleep(0.1) response = get_predictions(payload) del payload response_body = json.loads((response['Body'].read())) feature_lst = response_body['predictions'][0] return s3_uri, feature_lst

You define Amazon ES KNN index mapping with the following code:

#Define KNN Elasticsearch index mapping
knn_index = { "settings": { "index.knn": True }, "mappings": { "properties": { "zalando_img_vector": { "type": "knn_vector", "dimension": 2048 } } }

Import the image feature vector and associated Amazon S3 image URI into the Amazon ES KNN Index with the following code:

# defining a function to import the feature vectors corrosponds to each S3 URI into Elasticsearch KNN index
# This process will take around ~3 min. def es_import(i): es.index(index='idx_zalando', body={"zalando_img_vector": i[1], "image": i[0]} ) process_map(es_import, result, max_workers=workers)

Building a full-stack visual search application

Now that you have a working Amazon SageMaker endpoint for extracting image features and a KNN index on Amazon ES, you’re ready to build a real-world full-stack ML-powered web app. You use an AWS SAM template to deploy a serverless REST API with API Gateway and Lambda. The REST API accepts new images, generates the embeddings, and returns similar images to the client. Then you upload a front-end website that interacts with your new REST API to Amazon S3. The front-end code uses Amplify to integrate with your REST API.

  1. In the following cell, prepopulate a CloudFormation template that creates necessary resources such as Lambda and API Gateway for full-stack application:
    s3_resource.Object(bucket, 'backend/template.yaml').upload_file('./backend/template.yaml', ExtraArgs={'ACL':'public-read'}) sam_template_url = f'https://{bucket}' # Generate the CloudFormation Quick Create Link print("Click the URL below to create the backend API for visual search:n")
    print(( '' f'?templateURL={sam_template_url}' '&stackName=vis-search-api' f'&param_BucketName={outputs["s3BucketTraining"]}' f'&param_DomainName={outputs["esDomainName"]}' f'&param_ElasticSearchURL={outputs["esHostName"]}' f'&param_SagemakerEndpoint={predictor.endpoint}'

    The following screenshot shows the output: a pre-generated CloudFormation template link.

  2. Choose the link.

You are sent to the Quick create stack page.

  1. Select the check boxes to acknowledge the creation of IAM resources, IAM resources with custom names, and CAPABILITY_AUTO_EXPAND.
  2. Choose Create stack.

After the stack creation is complete, you see the status CREATE_COMPLETE. You can look on the Resources tab to see all the resources the CloudFormation template created.

  1. After the stack is created, proceed through the cells.

The following cell indicates that your full-stack application, including front-end and back-end code, are successfully deployed:

print('Click the URL below:n')
print(outputs['S3BucketSecureURL'] + '/index.html')

The following screenshot shows the URL output.

  1. Choose the link.

You are sent to the application page, where you can upload an image of a dress or provide the URL link of a dress and get similar dresses.

  1. When you’re done testing and experimenting with your visual search application, run the last two cells at the bottom of the notebook:
    # Delete the endpoint
    predictor.delete_endpoint() # Empty S3 Contents
    training_bucket_resource = s3_resource.Bucket(bucket)
    training_bucket_resource.objects.all().delete() hosting_bucket_resource = s3_resource.Bucket(outputs['s3BucketHostingBucketName'])

    These cells terminate your Amazon SageMaker endpoint and empty your S3 buckets to prepare you for cleaning up your resources.

Cleaning up

To delete the rest of your AWS resources, go to the AWS CloudFormation console and delete the vis-search-api and vis-search stacks.


In this post, we showed you how to create an ML-based visual search application using Amazon SageMaker and the Amazon ES KNN index. You used a pre-trained Resnet50 model trained on an ImageNet dataset. However, you can also use other pre-trained models, such as VGG, Inception, and MobileNet, and fine-tune with your own dataset.

A GPU instance is recommended for most deep learning purposes. Training new models is faster on a GPU instance than a CPU instance. You can scale sub-linearly when you have multi-GPU instances or if you use distributed training across many instances with GPUs. However, we used CPU instances for this use case so that you can complete the walkthrough under the AWS Free Tier.

For more information about the code sample in the post, see the GitHub repo. For more information about Amazon ES, see the following:

About the Authors

Amit Mukherjee is a Sr. Partner Solutions Architect with AWS. He provides architectural guidance to help partners achieve success in the cloud. He has a special interest in AI and machine learning. In his spare time, he enjoys spending quality time with his family.

Laith Al-Saadoon is a Sr. Solutions Architect with a focus on data analytics at AWS. He spends his days obsessing over designing customer architectures to process enormous amounts of data at scale. In his free time, he follows the latest in machine learning and artificial intelligence.



Listen: OakNorth CIO shares automation trends in commercial lending




Commercial banks have been automating aspects of the lending and decisioning process, primarily at the lower end of the commercial lending spectrum, but hesitate to automate for loans more than $1 million. This means commercial banks have kept automations focused on loans of less than $1 million, explains Sean Hunter in this podcast discussion with […]

Coinsmart. Beste Bitcoin-Börse in Europa

Continue Reading


Predictive Maintenance is a Killer AI App 




Predictive maintenance resulting from IoT and AI working together has been identified as a killer app, with a track record of ROI. (Credit: Getty Images) 

By John P. Desmond, AI Trends Editor 

Predictive maintenance (PdM) has emerged as a killer AI app. 

In the past five years, predictive maintenance has moved from a niche use case to a fast-growing, high return on investment (ROI) application that is delivering true value to users. These developments are an indication of the power of the Internet of Things (IoT) and AI together, a market considered in its infancy today. 

These observations are from research conducted by IoT Analytics, consultants who supply market intelligence, which recently estimated that the $6.9 billion predictive maintenance market will reach $28.2 billion by 2026.  

The company began its research coverage of the IoT-driven predictive maintenance market in 2016, at an industry maintenance conference in Dortmund, Germany. Not much was happening. “We were bitterly disappointed,” stated Knud Lasse Lueth, CEO at IoT Analytics, in an account in IoT Business News. “Not a single exhibitor was talking about predictive maintenance.”  

Things have changed. IoT Analytics analyst Fernando Alberto Brügge stated, Our research in 2021 shows that predictive maintenance has clearly evolved from the rather static condition-monitoring approach. It has become a viable IoT application that is delivering overwhelmingly positive ROI.” 

Technical developments that have contributed to the market expansion include: a simplified process for connecting IoT assets, major advances in cloud services, and improvements in the accessibility of machine learning/data science frameworks, the analysts state.  

Along with the technical developments, the predictive maintenance market has seen a steady increase in the number of software and service providers offering solutions. IoT Analytics identified about 100 companies in the space in 2016; today the company identifies 280 related solution providers worldwide. Many of them are startups who recently entered the field. Established providers including GE, PTC, Cisco, ABB, and Siemens, have entered the market in the past five years, many through acquisitions.  

The market still has room; the analysts predict 500 companies will be in the business in the next five years.  

In 2016, the ROI from predictive analytics was unclear. In 2021, a survey of about 100 senior IT executives from the industrial sector found that predictive maintenance projects have delivered a positive ROI in 83% of the cases. Some 45% of those reported amortizing their investments in less than a year. “This data demonstrated how attractive the investment has become in recent years,” the analysts stated.   

More IoT Sensors Means More Precision 

Implemented projects that the analysts studied in 2016 relied on a limited number of data sources, typically one sensor value, such as vibration or temperature. Projects described in the 2021 report described 11 classes of data sources, such as data from existing sensors or data from the controllers. As more sources are tapped, the precision of the predictions increase, the analysts state.  

Many projects today are using hybrid modeling approaches that rely on domain expertise, virtual sensors and augmented data. AspenTech and PARC are two suppliers identified in the report as embracing hybrid modeling approaches. AspenTech has worked with over 60 companies to develop and test hybrid models that combine physics with ML/data science knowledge, enhancing prediction accuracy. 

The move to edge computing is expected to further benefit predictive modeling projects, by enabling algorithms to run at the point where data is collected, reducing response latency. The supplier STMicroelectronics recently introduced some smart sensor nodes that can gather data and do some analytic processing. 

More predictive maintenance apps are being integrated with enterprise software systems, such as enterprise resource planning (ERP) or computerized maintenance  management systems (CMMS). Litmus Automation offers an integration service to link to any industrial asset, such as a programmable logic controller, a distributed control system, or a supervisory control and data acquisition system.   

Reduced Downtime Results in Savings 

Gains come from preventing downtime. Predictive maintenance is the result of monitoring operational equipment and taking action to prevent potential downtime or an unexpected or negative outcome,” stated Mike Leone, an analyst at IT strategy firm Enterprise Strategy Group, in an account from TechTarget.  

Felipe Parages, Senior Data Scientist, Valkyrie

Advances that have made predictive maintenance more practical today include sensor technology becoming more widespread, and the ability to monitor industrial machines in real time, stated Felipe Parages, senior data scientist at Valkyrie, data sense consultants. With more sensors, the volume of data has grown exponentially, and data analytics via cloud services has become available. 

It used to be that an expert had to perform an analysis to determine if a machine was not operating in an optimal way. “Nowadays, with the amount of data you can leverage and the new techniques based on machine learning and AI, it is possible to find patterns in all that data, things that are very subtle and would have escaped notice by a human being,” stated Parages. 

As a result, one person can now monitor hundreds of machines, and companies are accumulating historical data, which enables deeper trend analysis. “Predictive maintenance “is a very powerful weapon,” he stated.  

In an example project, Italy’s primary rail operator, Trenitalia, adopted predictive maintenance for its high-speed trains. The system is expected to save eight to 10% of an annual maintenance budget of 1.3 billion Euros, stated Paul Miller, an analyst with research firm Forrester, which recently issued a report on the project.  

They can eliminate unplanned failures which often provide direct savings in maintenance but just as importantly, by taking a train out of service before it breaks—that means better customer service and happier customers,” Miller stated. He recommended organizations start out with predictive maintenance by fielding a pilot project. 

In an example of the types of cooperation predictive maintenance projects are expected to engender, the CEOs of several European auto and electronics firms recently announced plans to join forces to form the “Software Republique,” a new ecosystem for innovation in intelligent mobility. Atos, Dassault Systèmes, Groupe Renault, and STMicroelectronics and Thales announced their decision to pool their expertise to accelerate the market.   

Luca de Meo, Chief Executive Officer, Groupe Renault

Luca de Meo, Chief Executive Officer of Groupe Renault, stated in a press release from STMicroelectronics, In the new mobility value chain, on-board intelligence systems are the new driving force, where all research and investment are now concentrated. Faced with this technological challenge, we are choosing to play collectively and openly. There will be no center of gravity, the value of each will be multiplied by others. The combined expertise in cybersecurity, microelectronics, energy and data management will enable us to develop unique, cutting-edge solutions for low-carbon, shared, and responsible mobility, made in Europe.”    

The Software République will be based in Guyancourt, a commune in north-central France at the Renault Technocentre in a building called Odyssée, a 12,000 square meter space which is eco-responsible. For example, its interior and exterior structure is 100 percent wood, and the building is covered with photovoltaic panels. 

Read the source articles in IoT Business News TechTarget, and in a press release from STMicroelectronics.

Coinsmart. Beste Bitcoin-Börse in Europa

Continue Reading


Post Office Looks to Gain an Edge With Edge Computing 




By AI Trends Editor John P. Desmond  

NVIDIA on May 6 detailed a partnership with the US Postal Service underway for over a year to speed up mail service using AI, with a goal of reducing current processing time tasks that take days to hours.   

The project fields edge servers at 195 Post Services sites across the nation, which review 20 terabytes of images a day from 1,000 mail processing machines, according to a post on the NVIDIA blog.  

Anthony Robbins, Vice President of Federal, Nvidia

“The federal government has been for the last several years talking about the importance of artificial intelligence as a strategic imperative to our nation, and as an important funding priority. It’s been talked about in the White House, on Capitol Hill, in the Pentagon. It’s been funded by billions of dollars, and it’s full of proof of concepts and pilots,” stated Anthony Robbins, Vice President of Federal for NVIDIA, in an interview with Nextgov “And this is one of the few enterprisewide examples of an artificial intelligence deployment that I think can serve to inspire the whole of the federal government.”  

The project started with USPS AI architect at the time Ryan Simpson, who had the idea to try to expand an image analysis system a postal team was developing, into something much bigger, according to the blog post. (Simpson worked for USPS for over 12 years, and moved to NVIDIA as a senior data scientist eight months ago.) He believed that a system could analyze billions of images each center generated, and gain insights expressed in a few data points that could be shared quickly over the network.  

In a three-week sprint, Simpson worked with half a dozen architects at NVIDIA and others to design the needed deep-learning models. The work was done within the Edge Computing Infrastructure Program (ECIP), a distributed edge AI system up and running on Nvidia’s EGX platform at USPS. The EGX platform enables existing and modern, data-intensive applications to be accelerated and secure on the same infrastructure, from data center to edge. 

“It used to take eight or 10 people several days to track down items, now it takes one or two people a couple of hours,” stated Todd Schimmel, Manager, Letter Mail Technology, USPS. He oversees USPS systems including ECIP, which uses NVIDIA-Certified edge servers from Hewlett-Packard Enterprise.  

In another analysis, a computer vision task that would have required two weeks on a network of servers with 800 CPUs can now get done in 20 minutes on the four NVIDIA V100 Tensor Core GPUs in one of the HPE Apollo 6500 servers.  

Contract Awarded in 2019 for System Using OCR  

USPS had put out a request for proposals for a system using optical character recognition (OCR) to streamline its imaging workflow. “In the past, we would have bought new hardware, software—a whole infrastructure for OCR; or if we used a public cloud service, we’d have to get images to the cloud, which takes a lot of bandwidth and has significant costs when you’re talking about approximately a billion images,” stated Schimmel. 

AI algorithms were developed on these NVIDIA DGX servers at a US Postal Service Engineering facility. (Credit: Nvidia)

Today, the new OCR application will rely on a deep learning model in a container on ECIP managed by Kubernetes, the open source container orchestration system, and served by NVIDIA Triton, the company’s open-source inference-serving software. Triton allows teams to deploy trained AI models from any framework, such as TensorFlow or PyTorch. 

The deployment was very streamlined,” Schimmel stated. “We awarded the contract in September 2019, started deploying systems in February 2020 and finished most of the hardware by August—the USPS was very happy with that,” he added 

Multiple models need to communicate to the USPS OCR application to work. The app that checks for mail items alone requires coordinating the work of more than a half dozen deep-learning models, each checking for specific features. And operators expect to enhance the app with more models enabling more features in the future. 

“The models we have deployed so far help manage the mail and the Postal Service—they help us maintain our mission,” Schimmel stated.  

One model, for example, automatically checks to see if a package carries the right postage for its size, weight, and destination. Another one that will automatically decipher a damaged barcode could be online this summer.  

“We’re at the very beginning of our journey with edge AI. Every day, people in our organization are thinking of new ways to apply machine learning to new facets of robotics, data processing and image handling,” he stated. 

Accenture Federal Services, Dell Technologies, and Hewlett-Packard Enterprise contributed to the USPS OCR system incorporating AI, Robbins of NVIDIA stated. Specialized computing cabinets—or nodes—that contain hardware and software specifically tuned for creating and training ML models, were installed at two data centers.   

The AI work that has to happen across the federal government is a giant team sport,” Robbins stated to Nextgov. “And the Postal Service’s deployment of AI across their enterprise exhibited just that.” 

The new solutions could help the Postal Service improve delivery standards, which have fallen over the past year. In mid-December, during the last holiday season, the agency delivered as little as 62% of first-class mail on time—the lowest level in years, according to an account in VentureBeat . The rate rebounded to 84% by the week of March 6 but remained below the agency’s target of about 96%. 

The Postal Service has blamed the pandemic and record peak periods for much of the poor service performance. 

Read the source articles and information on the Nvidia blog, in Nextgov and in VentureBeat.

Coinsmart. Beste Bitcoin-Börse in Europa

Continue Reading


Here Come the AI Regulations  




New proposed laws to govern AI are being entertained in the US and Europe, with China following a government-first approach. (Credit: Getty Images)  

By AI Trends Staff 

New laws will soon shape how companies use AI.   

The five largest federal financial regulators in the US recently released a request for information how banks use AI, signaling that new guidance is coming for the finance business. Soon after that, the US Federal Trade Commission released a set of guidelines on “truth, fairness and equity” in AI, defining the illegal use of AI as any act that “causes more harm than good,” according to a recent account in Harvard Business Review  

And on April 21, the European Commission issued its own proposal for the regulation of AI (See AI Trends, April 22, 2021)  

Andrew Burt, Managing Partner,

While we don’t know what these regulation will allow, “Three central trends unite nearly all current and proposed laws on AI, which means that there are concrete actions companies can undertake right now to ensure their systems don’t run afoul of any existing and future laws and regulations,” stated article author Andrew Burt, the managing partner of, a boutique law firm focused on AI and analytics.  

First, conduct assessments of AI risks. As part of the effort, document how the risks have been minimized or resolved. Regulatory frameworks that refer to these “algorithmic impact assessments,” or “IA for AI,” are available.  

For example, Virginia’s recently-passed Consumer Data Protection Act, requires assessments for certain types of high-risk algorithms. 

The EU’s new proposal requires an eight-part technical document to be completed for high-risk AI systems that outlines “the foreseeable unintended outcomes and sources of risks” of each AI system, Burt states. The EU proposal is similar to the Algorithmic Accountability Act filed in the US Congress in 2019. The bill did not go anywhere but is expected to be reintroduced.  

Second, accountability and independence. This suggestion is that the data scientists, lawyers and others evaluating the AI system have different incentives than those of the frontline data scientists. This could mean that the AI is tested and validated by different technical personnel than those who originally developed it, or organizations may choose to hire outside experts to assess the AI system.   

“Ensuring that clear processes create independence between the developers and those evaluating the systems for risk is a central component of nearly all new regulatory frameworks on AI,” Burt states.  

Third, continuous review. AI systems are “brittle and subject to high rates of failure,” with risks that grow and change over time, making it difficult to mitigate risk at a single point in time. “Lawmakers and regulators alike are sending the message that risk management is a continual process,” Burt stated.  

Approaches in US, Europe and China Differ  

The approaches between the US, Europe and China toward AI regulation differ in their approach, according to a recent account in The Verdict, based on analysis by Global Data, the data analytics and consulting company based in London. 

“Europe appears more optimistic about the benefits of regulation, while the US has warned of the dangers of over regulation,”’ the account states. Meanwhile, “China continues to follow a government-first approach” and has been widely criticized for the use of AI technology to monitor citizens. The account noted examples in the rollout by Tencent last year of an AI-based credit scoring system to determine the “trust value” of people, and the installation of surveillance cameras outside people’s homes to monitor the quarantine imposed after the breakout of COVID-19. 

Whether the US’ tech industry-led efforts, China’s government-first approach, or Europe’s privacy and regulation-driven approach is the best way forward remains to be seen,” the account stated. 

In the US, many companies are aware of the risk of new AI regulation that could stifle innovation and their ability to grow in the digital economy, suggested a recent report from pwc, the multinational professional services firm.   

It’s in a company’s interests to tackle risks related to data, governance, outputs, reporting, machine learning and AI models, ahead of regulation,” the pwc analysts state. They recommended business leaders assemble people from across the organization to oversee accountability and governance of technology, with oversight from a diverse team that includes members with business, IT and specialized AI skills.  

Critics of European AI Act Cite Too Much Gray Area 

While some argue that the European Commission’s proposed AI Act leaves too much gray area, the hope of the European Commission is that their proposed AI Act will provide guidance for businesses wanting to pursue AI, as well as a degree of legal certainty.   

Thierry Breton, European Commissioner for the Internal Market

“Trust… we think is vitally important to allow the development we want of artificial intelligence,” stated Thierry Breton, European Commissioner for the Internal Market, in an account in TechCrunch. AI applications “need to be trustworthy, safe, non-discriminatory — that is absolutely crucial — but of course we also need to be able to understand how exactly these applications will work.” 

“What we need is to have guidance. Especially in a new technology… We are, we will be, the first continent where we will give guidelines—we’ll say ‘hey, this is green, this is dark green, this is maybe a little bit orange and this is forbidden’. So now if you want to use artificial intelligence applications, go to Europe! You will know what to do, you will know how to do it, you will have partners who understand pretty well and, by the way, you will come also to the continent where you will have the largest amount of industrial data created on the planet for the next ten years.” 

“So come here—because artificial intelligence is about data—we’ll give you the guidelines. We will also have the tools to do it and the infrastructure,” Breton suggested. 

Another reaction was that the Commission’s proposal has overly broad exemptions, such as for law enforcement to use remote biometric surveillance including facial recognition technology, and it does not go far enough to address the risk of discrimination. 

Reactions to the Commission’s proposal included plenty of criticism of overly broad exemptions for law enforcement’s use of remote biometric surveillance (such as facial recognition tech) as well as concerns that measures in the regulation to address the risk of AI systems discriminating don’t go nearly far enough. 

“The legislation lacks any safeguards against discrimination, while the wide-ranging exemption for ‘safeguarding public security’ completely undercuts what little safeguards there are in relation to criminal justice,” stated Griff Ferris, legal and policy officer for Fair Trials, the global criminal justice watchdog based in London. “The framework must include rigorous safeguards and restrictions to prevent discrimination and protect the right to a fair trial. This should include restricting the use of systems that attempt to profile people and predict the risk of criminality.”  

To accomplish this, he suggested, “The EU’s proposals need radical changes to prevent the hard-wiring of discrimination in criminal justice outcomes, protect the presumption of innocence and ensure meaningful accountability for AI in criminal justice. 

Read the source articles and information in Harvard Business Review, in The Verdict and in TechCrunch. 

Coinsmart. Beste Bitcoin-Börse in Europa

Continue Reading
AI2 days ago

Build a cognitive search and a health knowledge graph using AWS AI services

Energy3 days ago

ONE Gas to Participate in American Gas Association Financial Forum

Blockchain1 day ago

Shiba Inu: Know How to Buy the New Dogecoin Rival

SaaS5 days ago

Blockchain5 days ago

Yieldly announces IDO

Blockchain2 days ago

Meme Coins Craze Attracting Money Behind Fall of Bitcoin

Blockchain5 days ago

Opimas estimates that over US$190 billion worth of Bitcoin is currently at risk due to subpar safekeeping

Esports3 days ago

Pokémon Go Special Weekend announced, features global partners like Verizon, 7-Eleven Mexico, and Yoshinoya

SaaS5 days ago

Fintech3 days ago

Credit Karma Launches Instant Karma Rewards

Esports2 days ago

‘Destroy Sandcastles’ in Fortnite Locations Explained

Esports2 days ago

Valve launches Supporters Clubs, allows fans to directly support Dota Pro Circuit teams

Blockchain2 days ago

Sentiment Flippening: Why This Bitcoin Expert Doesn’t Own Ethereum

SaaS5 days ago

Business Insider3 days ago

Bella Aurora launches its first treatment for white patches on the skin

Esports3 days ago

How to download PUBG Mobile’s patch 1.4 update

Esports4 days ago

5 Best Mid Laners in League of Legends Patch 11.10

Cyber Security4 days ago

Top Tips On Why And How To Get A Cyber Security Degree ?

Blockchain5 days ago

Decentraland Price Prediction 2021-2025: MANA $25 by the End of 2025

Private Equity3 days ago

Warburg Pincus leads $110m Aetion Series C in wake of company doubling revenue last year