Connect with us


Bitcoin IRA Accounts Are Soaring as Bitcoin Passes $18,000





Plato had Big Data and AI firmly on his radar




Imagine spending your life chained to a cave wall with your face to the wall.

First to set the scene and make this easy to follow I’ll make just two short points:

AI without a lot of data is really not AI, Its just a rulebook. The self-driving car is an example. No different to any app you use, but a bit more complicated. Just because it used to be done by humans doesn’t make it intelligent anymore than Word processing or CRM.

a. Big data is a huge weak point in AI because it is never and most likely never will be accurate or sufficiently complete.

b. The inexplicability of the rules AI extracts from bodies of data makes it too dangerous for most situations until that problem is solved.

Plato’s allegory of the cave was a ‘form’ in which he described Big Data and other similar concepts. Plato’s ‘forms’ are capable of adapting themselves to concepts not yet thought of because that is in essence, how they work.

Plato’s Cave allegory as narrated through Socrates describes vividly a tribe of people who spend their lives chained to a cave wall with their faces to the wall.

Behind them, life goes on as normal around a camp-fire.
These people see the world through the shadows cast on the cave wall. People, things, movements, even emotions all play out before them on the cave wall as shadows.

These people have a strong sense of life, but their sense is a shadow of the real thing at best. Key details are completely missing, a detail that might have completely changed their interpretation of the drama playing out on the wall before them. Their sense of scale, speed, and the relationships between one form and another, all are misinterpreted. Vital information that we all take for granted and yet continue to make terrible mistakes frequently are entirely missing for these unfortunate people chained to the cave wall.
If only we could let them view the scene from behind via a tv camera, let them see the expressions on the faces of speakers and recipients, know what lives in the hearts of the believers and of the non-believers. Maybe their view of that hidden world would be more accurate. Maybe then they would be better informed and make better decisions.

Plato was describing AI

The man facing the wall hears the sounds and sees the shadows of movement. His mind has to make decisions about what it is that he is hearing and seeing. The canvas on which he must paint this picture is full of his fears and desires based on past experiences and shared knowledge of others along the wall. Both the current situation and the canvas are utterly distorted and the outcome is usually barely salient. “A large and growing shadow approaching the wall outside of mealtimes means an imminent beating.”. That explains a past experience, but has no concept of the effect of distance from the fire, targets in between, or indeed all the other reasons why someone big or small might approach the wall.

Even when he is eventually free to turn around and his eyes are able to stand the glare, he will have little frame of reference with which to begin absorbing the truth about life, even assuming he has such a desire. Rules once created, rarely have any mechanism to be removed and he is stuck forever with the majority of his mistakes from the past. Soon his behaviors will begin to earn him beatings and starvation and his ill-conceived rules will begin to prove to him that he was right all along.

How shadows work in AI.
The bulk of AI, especially if it does anything useful is based on Machine Learning. Almost everything else is not AI, but a rulebook, i.e. a human told it the rules to follow. That’s how M.S. Word works and it’s not AI.
Another way is human classification or human/machine classification whereby a machine suggests classifications for something and a human checks it is safe before agreeing or saying no. That is a safer but less powerful approach because of the cost and the fact that shadows are not allowed. Humans don’t understand them.

Machine learning simply teaches the machine to look at what, to you and I, is a great fog or pea soup and spot groups of things that are repeated. This thing it discovers may be called an “AQ138_:wer” for argument’s sake. The machine doesn’t care. It’s a shadow catcher.

Then we give the machine a “training set”, say a collection of dog pictures and we ask it to compare the data sets awhile labeling all the groupings/ shadows that match a dog picture as “dog”. If we get results, we have achieved something. Now it is able to find dog shadows. They could be Wolves or possibly even cats however, there’s a lot of work to be done yet with more refined training sets.

One example I came across was successfully finding wolves that were intentionally scattered among dog pictures until it was discovered that the recognition factor was a patch of snow in the picture. Our friend had not noticed that most of the training set for wolves contained snow. Shadows are dangerous things even if at times they are exciting.
It took a great deal of trial and error in this example, through masking different parts of the training pictures for subsequent tests before it was possible to discover that there was an error and why the error was happening. This illustrates perfectly the problem of inexplicability with Machine learning and AI. Not knowing there is an error is far more dangerous than the challenges of finding the cause and eliminating the error. The truth is, we don’t know that the errors exist, or at least where they are. Imagine such a machine flying an aircraft full of people.

Or perhaps Plato was talking about Social Media

It may have occurred to you while we were poking fun at machines, that we humans are making very similar mistakes every day as we stare at our phones and PCs to see who is saying what on Social Media.
Now the whole gambit of this subject is for another day, but the bit that fits right into our discussion is the shadow on the wall that represents our friend, or the person we think is our friend that posted a picture and a profile online and told us we were clever.
The people, from friends to fakes to bots run by PR firms or even governments are all shadows of the real world no more or less than the shadows around the campfire.

The difference here is that these shadows are deliberately manipulated by their owners to show you a particular shadow, not just any random shadow, let alone a truthful or representative one.
What is to be learned from these shadows in the sense of individuals are more likely to be the thing each owner wants to project and hence that which they are not, or not at the level they aspire to.

In the sense of organizations and governments, the shadows represent a charade or mask of some sort representing that which they want you to believe but are almost certainly not what it really is.
How these shadows are presented to you, will tell you the opinion they have of you. Mostly this is an opinion on your type rather than you as an individual and indeed it reveals their view of the world more than yours.
Then of course there’s the platform, the guy who is earning from the campfire and needs you to remain chained to the wall. He makes sure that you get the feedback you need to keep recognizing and responding to the shadows and wanting to see more of them.

Unlike Plato’s prisoners, these prisoners are self-imprisoned. They are able to walk out of the cave and observe the campfire, yet they choose to relate to the shadow rather than the object. This is precise as Plato hypothesized.

Plato’s prophesy for you and me is terrifying if only we are only able to see it.

Today we live in a world where all of our news and our interactions with the world we live in are experienced via Social Media Channels and often second, or third-hand, or further and edited/presented with the attached attitudes of our peers. In choosing what we read, or view we are choosing our peers and vice versa. This is powerful and highly restricting, even more so than Plato’s cave allegory.

Indeed, traditional sources of news and information, what is left of them, are rapidly withering. Disappearing, or succumbing to the same fate. Soon there won’t be a campfire to see, because that too has been put in place for our benefit, and reality is hidden even deeper, should we have the desire or the constitution to look directly upon such a blinding light or any urge at all to depart the wall.

The people who govern us and police us, long ago will have ceased to be people, but AI-driven machines that make decisions based on the average of what the shadows add up to, but much more on the basis of which way will cost less and yield more for whoever runs the show.

Right now, it looks increasingly as though the owners of tech platforms from Social Media to Search and cloud AI algorithms are the people holding the aces. Theirs is the campfire and theirs is the lens that sees the shadows and tells us what to think about them.
The people who empower them are convinced that they will always be able to exert the power of the men who control the machines.

Source: Ed has enjoyed a dual career moving backward and forwards between leadership roles in software engineering and transforming marketing functions.

Continue Reading


This Week’s Awesome Tech Stories From Around the Web (Through January 23)





This Chinese Lab Is Aiming for Big AI Breakthroughs
Will Knight | Wired
“China produces as many artificial intelligence researchers as the US, but it lags in key fields like machine learning. The government hopes to make up ground. …It set AI researchers the goal of making ‘fundamental breakthroughs by 2025’ and called for the country to be ‘the world’s primary innovation center by 2030.’ BAAI opened a year later, in Zhongguancun, a neighborhood of Beijing designed to replicate US innovation hubs such as Boston and Silicon Valley.”


What Elon Musk’s $100 Million Carbon Capture Prize Could Mean
James Temple | MIT Technology Review
“[Elon Musk] announced on Twitter that he plans to give away $100 million of [his $180 billion net worth] as a prize for the ‘best carbon capture technology.’ …Another $100 million could certainly help whatever venture, or ventures, clinch Musk’s prize. But it’s a tiny fraction of his wealth and will also only go so far. …Money aside, however, one thing Musk has a particular knack for is generating attention. And this is a space in need of it.”


Synthetic Cornea Helped a Legally Blind Man Regain His Sight
Steve Dent | Engadget
“While the implant doesn’t contain any electronics, it could help more people than any robotic eye. ‘After years of hard work, seeing a colleague implant the CorNeat KPro with ease and witnessing a fellow human being regain his sight the following day was electrifying and emotionally moving, there were a lot of tears in the room,’ said CorNeat Vision co-founder Dr. Gilad Litvin.”


MIT Develops Method for Lab-Grown Plants That May Eventually Lead to Alternatives to Forestry and Farming
Darrell Etherington | TechCrunch
“If the work of these researchers can eventually be used to create a way to produce lab-grown wood for use in construction and fabrication in a way that’s scalable and efficient, then there’s tremendous potential in terms of reducing the impact on forestry globally. Eventually, the team even theorizes you could coax the growth of plant-based materials into specific target shapes, so you could also do some of the manufacturing in the lab, by growing a wood table directly for instance.”


FAA Approves First Fully Automated Commercial Drone Flights
Andy Pasztor and Katy Stech Ferek | The Wall Street Journal
“US aviation regulators have approved the first fully automated commercial drone flights, granting a small Massachusetts-based company permission to operate drones without hands-on piloting or direct observation by human controllers or observers. …The company’s Scout drones operate under predetermined flight programs and use acoustic technology to detect and avoid drones, birds, and other obstacles.”


China’s Surging Private Space Industry Is Out to Challenge the US
Neel V. Patel | MIT Technology Review
“[The Ceres-1] was a commercial rocket—only the second from a Chinese company ever to go into space. And the launch happened less than three years after the company was founded. The achievement is a milestone for China’s fledgling—but rapidly growing—private space industry, an increasingly critical part of the country’s quest to dethrone the US as the world’s preeminent space power.”


Janet Yellen Will Consider Limiting Use of Cryptocurrency
Timothy B. Lee | Ars Technica
“Cryptocurrencies could come under renewed regulatory scrutiny over the next four years if Janet Yellen, Joe Biden’s pick to lead the Treasury Department, gets her way. During Yellen’s Tuesday confirmation hearing before the Senate Finance Committee, Sen. Maggie Hassan (D-N.H.) asked Yellen about the use of cryptocurrency by terrorists and other criminals. ‘Cryptocurrencies are a particular concern,’ Yellen responded. ‘I think many are used—at least in a transactions sense—mainly for illicit financing.’i


Secret Ingredient Found to Power Supernovas
Thomas Lewton | Quanta
“…Only in the last few years, with the growth of supercomputers, have theorists had enough computing power to model massive stars with the complexity needed to achieve explosions. …These new simulations are giving researchers a better understanding of exactly how supernovas have shaped the universe we see today.”

Image Credit: Ricardo Gomez Angel / Unsplash


Continue Reading


Multi-account model deployment with Amazon SageMaker Pipelines




Amazon SageMaker Pipelines is the first purpose-built CI/CD service for machine learning (ML). It helps you build, automate, manage, and scale end-to-end ML workflows and apply DevOps best practices of CI/CD to ML (also known as MLOps).

Creating multiple accounts to organize all the resources of your organization is a good DevOps practice. A multi-account strategy is important not only to improve governance but also to increase security and control of the resources that support your organization’s business. This strategy allows many different teams inside your organization, to experiment, innovate, and integrate faster, while keeping the production environment safe and available for your customers.

Pipelines makes it easy to apply the same strategy to deploying ML models. Imagine a use case in which you have three different AWS accounts, one for each environment: data science, staging, and production. The data scientist has the freedom to run experiments and train and optimize different models any time in their own account. When a model is good enough to be deployed in production, the data scientist just needs to flip the model approval status to Approved. After that, an automated process deploys the model on the staging account. Here you can automate testing of the model with unit tests or integration tests or test the model manually. After a manual or automated approval, the model is deployed to the production account, which is a more tightly controlled environment used to serve inferences on real-world data. With Pipelines, you can implement a ready-to-use multi-account environment.

In this post, you learn how to use Pipelines to implement your own multi-account ML pipeline. First, you learn how to configure your environment and prepare it to use a predefined template as a SageMaker project for training and deploying a model in two different accounts: staging and production. Then, you see in detail how this custom template was created and how to create and customize templates for your own SageMaker projects.

Preparing the environment

In this section, you configure three different AWS accounts and use SageMaker Studio to create a project that integrates a CI/CD pipeline with the ML pipeline created by a data scientist. The following diagram shows the reference architecture of the environment that is created by the SageMaker custom project and how AWS Organizations integrates the different accounts.

The diagram contains three different accounts, managed by Organizations. Also, three different user roles (which may be the same person) operate this environment:

  • ML engineer – Responsible for provisioning the SageMaker Studio project that creates the CI/CD pipeline, model registry, and other resources
  • Data scientist – Responsible for creating the ML pipeline that ends with a trained model registered to the model group (also referred to as model package group)
  • Approver – Responsible for testing the model deployed to the staging account and approving the production deployment

It’s possible to run a similar solution without Organizations, if you prefer (although not recommended). But you need to prepare the permissions and the trust relationship between your accounts manually and modify the template to remove the Organizations dependency. Also, if you’re an enterprise with multiple AWS accounts and teams, it’s highly recommended that you use AWS Control Tower for provisioning the accounts and Organizations. AWS Control Tower provides the easiest way to set up and govern a new and secure multi-account AWS environment. For this post, we only discuss implementing the solution with Organizations.

But before you move on, you need to complete the following steps, which are detailed in the next sections:

  1. Create an AWS account to be used by the data scientists (data science account).
  2. Create and configure a SageMaker Studio domain in the data science account.
  3. Create two additional accounts for production and staging.
  4. Create an organizational structure using Organizations, then invite and integrate the additional accounts.
  5. Configure the permissions required to run the pipelines and deploy models on external accounts.
  6. Import the SageMaker project template for deploying models in multiple accounts and make it available for SageMaker Studio.

Configuring SageMaker Studio in your account

Pipelines provides built-in support for MLOps templates to make it easy for you to use CI/CD for your ML projects. These MLOps templates are defined as Amazon CloudFormation templates and published via AWS Service Catalog. These are made available to data scientists via SageMaker Studio, an IDE for ML. To configure Studio in your account, complete the following steps:

  1. Prepare your SageMaker Studio domain.
  2. Enable SageMaker project templates and SageMaker JumpStart for this account and Studio users.

If you have an existing domain, you can simply edit the settings for the domain or individual users to enable this option. Enabling this option creates two different AWS Identity and Account Management (IAM) roles in your AWS account:

  • AmazonSageMakerServiceCatalogProductsLaunchRole – Used by SageMaker to run the project templates and create the required infrastructure resources
  • AmazonSageMakerServiceCatalogProductsUseRole – Used by the CI/CD pipeline to run a job and deploy the models on the target accounts

If you created your SageMaker Studio domain before re:Invent 2020, it’s recommended that you refresh your environment by saving all the work in progress. On the File menu, choose Shutdown, and confirm your choice.

  1. Create and prepare two other AWS accounts for staging and production, if you don’t have them yet.

Configuring Organizations

You need to add the data science account and the two additional accounts to a structure in Organizations. Organizations helps you to centrally manage and govern your environment as you grow and scale your AWS resources. It’s free and benefits your governance strategy.

Each account must be added to a different organizational unit (OU).

  1. On the Organizations console, create a structure of OUs like the following:
  • Root
    • multi-account-deployment (OU)
      • 111111111111 (data science account—SageMaker Studio)
      • production (OU)
        • 222222222222 (AWS account)
      • staging (OU)
        • 333333333333 (AWS account)

After configuring the organization, each account owner receives an invite. The owners need to accept the invites, otherwise the accounts aren’t included in the organization.

  1. Now you need to enable trusted access with AWS organizations (“Enable all features” and “Enable trusted access in the StackSets”).

This process allows your data science account to provision resources in the target accounts. If you don’t do that, the deployment process fails. Also, this feature set is the preferred way to work with Organizations, and it includes consolidating billing features.

  1. Next, on the Organizations console, choose Organize accounts.
  2. Choose staging.
  3. Note down the OU ID.
  4. Repeat this process for the production OU.

Configuring the permissions

You need to create a SageMaker execution role in each additional account. These roles are assumed by AmazonSageMakerServiceCatalogProductsUseRole in the data science account to deploy the endpoints in the target accounts and test them.

  1. Sign in to the AWS Management Console with the staging account.
  2. Run the following CloudFormation template.

This template creates a new SageMaker role for you.

  1. Provide the following parameters:
    1. SageMakerRoleSuffix – A short string (maximum 10 lowercase with no spaces or alpha-numeric characters) that is added to the role name after the following prefix: sagemaker-role-. The final role name is sagemaker-role-<<sagemaker_role_suffix>>.
    2. PipelineExecutionRoleArn – The ARN of the role from the data science account that assumes the SageMaker role you’re creating. To find the ARN, sign in to the console with the data science account. On the IAM console, choose Roles and search for AmazonSageMakerServiceCatalogProductsUseRole. Choose this role and copy the ARN (arn:aws:iam::<<data_science_acccount_id>>:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole).
  2. After creating this role in the staging account, repeat this process for the production account.

In the data science account, you now configure the policy of the Amazon Simple Storage Service (Amazon S3) bucket used to store the trained model. For this post, we use the default SageMaker bucket of the current Region. It has the following name format: sagemaker-<<region>>-<<aws_account_id>>.

  1. On the Amazon S3 console, search for this bucket, providing the Region you’re using and the ID of the data science account.

If you don’t find it, create a new bucket following this name format.

  1. On the Permissions tab, add the following policy:
    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::<<staging_account_id>>:root", "arn:aws:iam::<<production_account_id>>:root" ] }, "Action": [ "s3:GetObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::sagemaker-<<region>>-<<aws_account_id>>", "arn:aws:s3:::sagemaker-<<region>>-<<aws_account_id>>/*" ] } ]

  1. Save your settings.

The target accounts now have permission to read the trained model during deployment.

The next step is to add new permissions to the roles AmazonSageMakerServiceCatalogProductsUseRole and AmazonSageMakerServiceCatalogProductsLaunchRole.

  1. In the data science account, on the IAM console, choose Roles.
  2. Find the AmazonSageMakerServiceCatalogProductsUseRole role and choose it.
  3. Add a new policy and enter the following JSON code.
  4. Save your changes.
  5. Now, find the AmazonSageMakerServiceCatalogProductsLaunchRole role, choose it and add a new policy with the following content:
    { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::aws-ml-blog/artifacts/sagemaker-pipeline-blog-resources/*" } ]

  1. Save your changes.

That’s it! Your environment is almost ready. You only need one more step and you can start training and deploying models in different accounts.

Importing the custom SageMaker Studio project template

In this step, you import your custom project template.

  1. Sign in to the console with the data science account.
  2. On the AWS Service Catalog console, under Administration, choose Portfolios.
  3. Choose Create a new portfolio.
  4. Name the portfolio SageMaker Organization Templates.
  5. Download the following template to your computer.
  6. Choose the new portfolio.
  7. Choose Upload a new product.
  8. For Product name¸ enter Multi Account Deployment.
  9. For Description, enter Multi account deployment project.
  10. For Owner, enter your name.
  11. Under Version details, for Method, choose Use a template file.
  12. Choose Upload a template.
  13. Upload the template you downloaded.
  14. For Version title, choose 1.0.

The remaining parameters are optional.

  1. Choose Review.
  2. Review your settings and choose Create product.
  3. Choose Refresh to list the new product.
  4. Choose the product you just created.
  5. On the Tags tab, add the following tag to the product:
    1. Keysagemaker:studio-visibility
    2. ValueTrue

Back in the portfolio details, you see something similar to the following screenshot (with different IDs).

  1. On the Constraints tab, choose Create constraint.
  2. For Product, choose Multi Account Deployment (the product you just created).
  3. For Constraint type, choose Launch.
  4. Under Launch Constraint, for Method, choose Select IAM role.
  5. Choose AmazonSageMakerServiceCatalogProductsLaunchRole.
  6. Choose Create.
  7. On the Groups, roles, and users tab, choose Add groups, roles, users.
  8. On the Roles tab, select the role you used when configuring your SageMaker Studio domain.
  9. Choose Add access.

If you don’t remember which role you selected, in your data science account, go to the SageMaker console and choose Amazon SageMaker Studio. In the Studio Summary section, locate the attribute Execution role. Search for the name of this role in the previous step.

You’re done! Now it’s time to create a project using this template.

Creating your project

In the previous sections, you prepared the multi-account environment. The next step is to create a project using your new template.

  1. Sign in to the console with the data science account.
  2. On the SageMaker console, open SageMaker Studio with your user.
  3. Choose the Components and registries
  4. On the drop-down menu, choose Projects.
  5. Choose Create project.

On the Create project page, SageMaker templates is chosen by default. This option lists the built-in templates. However, you want to use the template you prepared for the multi-account deployment.

  1. Choose Organization templates.
  2. Choose Multi Account Deployment.
  3. Choose Select project template.

If you can’t see the template, make sure you completed all the steps correctly in the previous section.

  1. In the Project details section, for Name, enter iris-multi-01.

The project name must have 15 characters or fewer.

  1. In the Project template parameters, use the names of the roles you created in each target account (staging and production) and provide the following properties:
    1. SageMakerExecutionRoleStagingName
    2. SageMakerExecutionRoleProdName
  2. Retrieve the OU IDs you created earlier for the staging and production OUs and provide the following properties:
    1. OrganizationalUnitStagingId
    2. OrganizationalUnitProdId
  3. Choose Create project.

Provisioning all the resources takes a few minutes, after which the project is listed in the Projects section. When you choose the project, a tab opens with the project’s metadata. The Model groups tab chows a model group with the same name as your project. It was also created during the project provisioning.

The environment is now ready for the data scientist to start training the model.

Training a model

Now that your project is ready, it’s time to train a model.

  1. Download the example notebook to use for this walkthrough.
  2. Choose the Folder icon to change the work area to file management.
  3. Choose the Create folder
  4. Enter a name for the folder.
  5. Choose the folder name.
  6. Choose the Upload file
  7. Choose the Jupyter notebook you downloaded and upload it to the new directory.
  8. Choose the notebook to open a new tab.

You’re prompted to choose a kernel.

  1. Choose Python3 (Data Science).
  2. Choose Select.

  1. In the second cell of the notebook, replace the project_name variable with the name you gave your project (for this post, iris-multi-01).

You can now run the Jupyter notebook. This notebook creates a very simple pipeline with only two steps: train and register model. It uses the iris dataset and the XGBoost built-in container as the algorithm.

  1. Run the whole notebook.

The process takes some time after you run the cell containing the following code:

start_response = pipeline.start(parameters={ "TrainingInstanceCount": "1"

This starts the training job, which takes approximately 3 minutes to complete. After the training is finished, the next cell of the Jupyter notebook gets the latest version of the model in the model registry and marks it as Approved. Alternatively, you can approve a model from the SageMaker Studio UI. On the Model groups tab, choose the model group and desired version. Choose Update status and Approve before saving.

This is the end of the data scientist’s job but the beginning of running the CI/CD pipeline.

Amazon EventBridge monitors the model registry. The listener starts a new deployment job with the provisioned AWS CodePipeline workflow (created with you launched the SageMaker Studio project).

  1. On the CodePipeline console, choose the pipeline starting with the prefix sagemaker-, followed by the name of your project.

Shortly after you approve your model, the deployment pipeline starts running. Wait for the pipeline to reach the state DeployStaging. That stage can take approximately 10 minutes to complete. After deploying the first endpoint in the staging account, the pipeline is tested, and then moves to the next step, ApproveDeployment. In this step, it waits for manual approval.

  1. Choose Review.
  2. Enter an approval reason in the text box.
  3. Choose Approve.

The model is now deployed in the production account.

You can also monitor the pipeline on the AWS CloudFormation console, to see the stacks and stack sets the pipeline creates to deploy endpoints in the target accounts. To see the deployed endpoints for each account, sign in to the SageMaker console as either the staging account or production account and choose Endpoints on the navigation pane.

Cleaning up

To clean up all the resources you provisioned in this example, complete the following steps:

  1. Sign in to the console with your main account.
  2. On the AWS CloudFormation console, click on StackSets and delete the following items (endpoints):
    1. Prod sagemaker-<<sagemaker-project-name>>-<<project-id>>-deploy-prod
    2. Stagingsagemaker-<<sagemaker-project-name>>-<<project-id>>-deploy-staging
  3. In your laptop or workstation terminal, use the AWS Command Line Interface (AWS CLI) and enter the following code to delete your project:
    aws sagemaker delete-project --project-name iris-multi-01

Make sure you’re using the latest version of the AWS CLI.

Building and customizing a template for your own SageMaker project

SageMaker projects and SageMaker MLOps project templates are powerful features that you can use to automatically create and configure the whole infrastructure required to train, optimize, evaluate, and deploy ML models. A SageMaker project is an AWS Service Catalog provisioned product that enables you to easily create an end-to-end ML solution. For more information, see the AWS Service Catalog Administrator Guide.

A product is a CloudFormation template managed by AWS Service Catalog. For more information about templates and their requirements, see AWS CloudFormation template formats.

ML engineers can design multiple environments and express all the details of this setup as a CloudFormation template, using the concept of infrastructure as code (IaC). You can also integrate these different environments and tasks using a CI/CD pipeline. SageMaker projects provide an easy, secure, and straightforward way of wrapping the infrastructure complexity around in the format of a simple project, which can be launched many times by the other ML engineers and data scientists.

The following diagram illustrates the main steps you need to complete in order to create and publish your custom SageMaker project template.

We described these steps in more detail in the sections Importing the custom SageMaker Studio Project template and Creating your project.

As an ML engineer, you can design and create a new CloudFormation template for the project, prepare an AWS Service Catalog portfolio, and add a new product to it.

Both data scientists and ML engineers can use SageMaker Studio to create a new project with the custom template. SageMaker invokes AWS Service Catalog and starts provisioning the infrastructure described in the CloudFormation template.

As a data scientist, you can now start training the model. After you register it in the model registry, the CI/CD pipeline runs automatically and deploys the model on the target accounts.

If you look at the CloudFormation template from this post in a text editor, you can see that it implements the architecture we outline in this post.

The following code is a snippet of the template:

Description: Toolchain template which provides the resources needed to represent infrastructure as code. This template specifically creates a CI/CD pipeline to deploy a given inference image and pretrained Model to two stages in CD -- staging and production.
Parameters: SageMakerProjectName: Type: String SageMakerProjectId: Type: String
<<other parameters>>
Resources: MlOpsArtifactsBucket: Type: AWS::S3::Bucket DeletionPolicy: Retain Properties: BucketName: …
… ModelDeployCodeCommitRepository: Type: AWS::CodeCommit::Repository Properties: RepositoryName: … RepositoryDescription: … Code: S3: Bucket: … Key: …
… ModelDeployBuildProject: Type: AWS::CodeBuild::Project
… ModelDeployPipeline: Type: AWS::CodePipeline::Pipeline

The template has two key sections: Parameters (input parameters of the template) and Resources. SageMaker project templates require that you add two input parameters to your template: SageMakerProjectName and SageMakerProjectId. These parameters are used internally by SageMaker Studio. You can add other parameters if needed.

In the Resources section of the snippet, you can see that it creates the following:

  • A new S3 bucket used by the CI/CD pipeline to store the intermediary artifacts passed from one stage to another.
  • An AWS CodeCommit repository to store the artifacts used during the deployment and testing stages.
  • An AWS CodeBuild project to get the artifacts, and validate and configure them for the project. In the multi-account template, this project also creates a new model registry, used by the CI/CD pipeline to deploy new models.
  • A CodePipeline workflow that orchestrates all the steps of the CI/CD pipelines.

Each time you register a new model to the model registry or push a new artifact to the CodeCommit repo, this CodePipeline workflow starts. These events are captured by an EventBridge rule, provisioned by the same template. The CI/CD pipeline contains the following stages:

  • Source – Reads the artifacts from the CodeCommit repository and shares with the other steps.
  • Build – Runs the CodeBuild project to do the following:
    • Verify if a model registry is already created, and create one if needed.
    • Prepare a new CloudFormation template that is used by the next two deployment stages.
  • DeployStaging – Contains the following components:
    • DeployResourcesStaging – Gets the CloudFormation template prepared in the Build step and deploys a new stack. This stack deploys a new SageMaker endpoint in the target account.
    • TestStaging – Invokes a second CodeBuild project that runs a custom Python script that tests the deployed endpoint.
    • ApproveDeployment – A manual approval step. If approved, it moves to the next stage to deploy an endpoint in production, or ends the workflow if not approved.
  • DeployProd – Similar to DeployStaging, it uses the same CloudFormation template but with different input parameters. It deploys a new SageMaker endpoint in the production account. 

You can start a new training process and register your model to the model registry associated with the SageMaker project. Use the Jupyter notebook provided in this post and customize your own ML pipeline to prepare your dataset and train, optimize, and test your models before deploying them. For more information about these features, see Automate MLOps with SageMaker Projects. For more Pipelines examples, see the GitHub repo.

Conclusions and next steps

In this post, you saw how to prepare your own environment to train and deploy ML models in multiple AWS accounts by using SageMaker Pipelines.

With SageMaker projects, the governance and security of your environment can be significantly improved if you start managing your ML projects as a library of SageMaker project templates.

As a next step, try to modify the SageMaker project template and customize it to address your organization’s needs. Add as many steps as you want and keep in mind that you can capture the CI/CD events and notify users or call other services to build comprehensive solutions.

About the Author

Samir Araújo is an AI/ML Solutions Architect at AWS. He helps customers creating AI/ML solutions solve their business challenges using the AWS platform. He has been working on several AI/ML projects related to computer vision, natural language processing, forecasting, ML at the edge, and more. He likes playing with hardware and automation projects in his free time, and he has a particular interest for robotics.


Continue Reading

Artificial Intelligence

Extra Crunch roundup: Digital health VC survey, edtech M&A, deep tech marketing, more




I had my first telehealth consultation last year, and there’s a high probability that you did, too. Since the pandemic began, consumer adoption of remote healthcare has increased 300%.

Speaking as an unvaccinated urban dweller: I’d rather speak to a nurse or doctor via my laptop than try to remain physically distanced on a bus or hailed ride traveling to/from their office.

Even after things return to (rolls eyes) normal, if I thought there was a reliable way to receive high-quality healthcare in my living room, I’d choose it.

Clearly, I’m not alone: a May 2020 McKinsey study pegged yearly domestic telehealth revenue at $3 billion before the coronavirus, but estimated that “up to $250 billion of current U.S. healthcare spend could potentially be virtualized” after the pandemic abates.

That’s a staggering number, but in a category that includes startups focused on sexual health, women’s health, pediatrics, mental health, data management and testing, it’s clear to see why digital-health funding topped more than $10 billion in the first three quarters of 2020.

Drawing from The TechCrunch List, reporter Sarah Buhr interviewed eight active health tech VCs to learn more about the companies and industry verticals that have captured their interest in 2021:

  • Bryan Roberts and Bob Kocher, partners, Venrock
  • Nan Li, managing director, Obvious Ventures
  • Elizabeth Yin, general partner, Hustle Fund
  • Christina Farr, principal investor and health tech lead, OMERS Ventures
  • Ursheet Parikh, partner, Mayfield Ventures
  • Nnamdi Okike, co-founder and managing partner, 645 Ventures
  • Emily Melton, founder and managing partner, Threshold Ventures

Full Extra Crunch articles are only available to members
Use discount code ECFriday to save 20% off a one- or two-year subscription

Since COVID-19 has renewed Washington’s focus on healthcare, many investors said they expect a friendly regulatory environment for telehealth in 2021. Additionally, healthcare providers are looking for ways to reduce costs and lower barriers for patients seeking behavioral support.

“Remote really does work,” said Elizabeth Yin, general partner at Hustle Fund.

We’ll cover digital health in more depth this year through additional surveys, vertical reporting, founder interviews and much more.

Thanks very much for reading Extra Crunch this week; I hope you have a relaxing weekend.

Walter Thompson
Senior Editor, TechCrunch

8 VCs agree: Behavioral support and remote visits make digital health a strong bet for 2021

Image Credits: Luis Alvarez (opens in a new window) / Getty Images

Lessons from Top Hat’s acquisition spree

Image Credits: Bryce Durbin

In the last year, edtech startup Top Hat acquired three publishing companies: Fountainhead Press, Bludoor and Nelson HigherEd.

Natasha Mascarenhas interviewed CEO and founder Mike Silagadze to learn more about his content acquisition strategy, but her story also discussed “some rumblings of consolidation and exits in edtech land.”

How VCs invested in Asia and Europe in 2020

Last year, U.S.-based VCs invested an average of $428 million each day in domestic startups, with much of the benefits flowing to fintech companies.

This morning, Alex Wilhelm examined Q4 VC totals for Europe, which had its lowest deal count since Q1 2019, despite a record $14.3 billion in investments.

Asia’s VC industry, which saw $25.2 billion invested across 1,398 deals is seeing “a muted recovery,” says Alex.

“Falling seed volume, lots of big rounds. That’s 2020 VC around the world in a nutshell.”

Decrypted: With more SolarWinds fallout, Biden picks his cybersecurity team

Image Credits: Treedeo (opens in a new window) / Getty Images

In this week’s Decrypted, security reporter Zack Whittaker covered the latest news in the unfolding SolarWinds espionage campaign, now revealed to have impacted the U.S. Bureau of Labor Statistics and Malwarebytes.

In other news, the controversy regarding WhatsApp’s privacy policy change appears to be driving users to encrypted messaging app Signal, Zack reported. Facebook has put changes at WhatsApp on hold “until it could figure out how to explain the change without losing millions of users,” apparently.

Hot IPOs hang onto gains as investors keep betting on tech

A big IPO debut is a juicy topic for a few news cycles, but because there’s always another unicorn ready to break free from its corral and leap into the public markets, it doesn’t leave a lot of time to reflect.

Alex studied companies like Lemonade, Airbnb and Affirm to see how well these IPO pop stars have retained their value. Not only have most held steady, “many have actually run up the score in the ensuing weeks,” he found.

Dear Sophie: What are Biden’s immigration changes?

Image Credits: Bryce Durbin / TechCrunch

Dear Sophie:

I work in HR for a tech firm. I understand that Biden is rolling out a new immigration plan today.

What is your sense as to how the new administration will change business, corporate and startup founder immigration to the U.S.?

—Free in Fremont

Hello, Extra Crunch community!

Image Credits: atakan (opens in a new window) / Getty Images

I began my career as an avid TechCrunch reader and remained one even when I joined as a writer, when I left to work on other things and now that I’ve returned to focus on better serving our community.

I’ve been chatting with some of the folks in our community and I’d love to talk to you, too. Nothing fancy, just 5-10 minutes of your time to hear more about what you want to see from us and get some feedback on what we’ve been doing so far.

If you would be so kind as to take a minute or two to fill out this form, I’ll drop you a note and hopefully we can have a chat about the future of the Extra Crunch community before we formally roll out some of the ideas we’re cooking up.

Drew Olanoff

In 2020, VCs invested $428m into US-based startups every day

Last year was a disaster across the board thanks to a global pandemic, economic uncertainty and widespread social and political upheaval.

But if you were involved in the private markets, however, 2020 had some very clear upside — VCs flowed $156.2 billion into U.S.-based startups, “or around $428 million for each day,” reports Alex Wilhelm.

“The huge sum of money, however, was itself dwarfed by the amount of liquidity that American startups generated, some $290.1 billion.”

Using data sourced from the National Venture Capital Association and PitchBook, Alex used Monday’s column to recap last year’s seed, early-stage and late-stage rounds.

How and when to build marketing teams at deep tech companies

Image Credits: Andy Roberts (opens in a new window) / Getty Images

Building a marketing team is one of the most opaque parts of spinning up a startup, but for a deep tech company, the stakes couldn’t be higher.

How can technical founders working on bleeding-edge technology find the right people to tell their story?

If you work at a post-revenue, early-stage deep tech startup (or know someone who does), this post explains when to hire a team, whether they’ll need prior industry experience, and how to source and evaluate talent.

Bustle CEO Bryan Goldberg explains his plans for taking the company public

Bustle Digital Group CEO Bryan Goldberg. Image Credits: Bustle Digital Group

Senior Writer Anthony Ha interviewed Bustle Digital Group CEO Bryan Goldberg to get his thoughts on the state of digital media.

Their conversation covered a lot of ground, but the biggest news it contained focuses on Goldberg’s short-term plans.

“Where do I want to see the company in three years? I want to see three things: I want to be public, I want to see us driving a lot of profits and I want it to be a lot bigger, because we’ve consolidated a lot of other publications,” he said.

It may not be as glamorous as D2C, but beauty tech is big money

Image Credits: Laia Divols Escude/EyeEm (opens in a new window) / Getty Images

The U.S. Federal Trade Commission is not a huge fan of personal-care D2C brands merging with traditional consumer product companies.

This month, razor startup Billie and Proctor & Gamble announced they were calling off their planned merger after the FTC filed suit.

For similar reasons, Edgewell Personal Care dropped its plans last year to buy Harry’s for $1.37 billion.

In a harsher regulatory environment, “the path to profitability has become a more important part of the startup story versus growth at all costs,” it seems.

Twilio CEO says wisdom lies with your developers

SAN FRANCISCO, CA – SEPTEMBER 12: Founder and CEO of Twilio Jeff Lawson speaks onstage during TechCrunch Disrupt SF 2016 at Pier 48 on September 12, 2016 in San Francisco, California. Image Credits: Steve Jennings/Getty Images for TechCrunch

Companies that build their own tools “tend to win the hearts, minds and wallets of their customers,” according to Twilio CEO Jeff Lawson.

In an interview with enterprise reporter Ron Miller for his new book, “Ask Your Developer,” Lawson says founders should use developer teams as a sounding board when making build-versus-buy decisions.

“Lawson’s basic philosophy in the book is that if you can build it, you should,” says Ron.


Continue Reading
Cyber Security4 days ago

Critical Cisco SD-WAN Bugs Allow RCE Attacks

Medical Devices4 days ago

Elcam Medical Joins Serenno Medical as Strategic Investor and Manufacturer of its Automatic Monitoring of Kidney Function Device

Blockchain3 days ago

TA: Ethereum Starts Recovery, Why ETH Could Face Resistance Near $1,250

SPACS4 days ago

Intel Chairman Gets Medtronic Backing for $750 Million SPAC IPO

PR Newswire4 days ago

Global Laboratory Information Management Systems Market (2020 to 2027) – Featuring Abbott Informatics, Accelerated Technology Laboratories & Autoscribe Informatics Among Others

SPAC Insiders5 days ago

Queen’s Gambit Growth Capital (GMBT.U) Prices Upsized $300M IPO

SPACS4 days ago

Payments Startup Payoneer in Merger Talks With SPAC

SPAC Insiders5 days ago

FoxWayne Enterprises Acquisition Corp. (FOXWU) Prices $50M IPO

NEWATLAS5 days ago

New Street Bob 114 heads Harley-Davidson’s 2021 lineup

SPACS4 days ago

Michael Moe, fresh from raising $225M for education-focused SPAC, set for another free Startup Bootcamp

NEWATLAS5 days ago

World-first biomarker test can predict depression and bipolar disorder

Blockchain3 days ago

Bitcoin Cash Analysis: Strong Support Forming Near $400

Amb Crypto1 day ago

Bitcoin Cash, Synthetix, Dash Price Analysis: 23 January

Blockchain1 day ago

Buying the Bitcoin Dip: MicroStrategy Scoops $10M Worth of BTC Following $7K Daily Crash

Blockchain2 days ago

OIO Holdings Appoints Rudy Lim as CEO of Blockchain Business Subsidiary

SPACS4 days ago

Goldman CEO Says SPAC Explosion ‘Unsustainable’

Amb Crypto1 day ago

Why has Bitcoin’s brief recovery not been enough

NEWATLAS5 days ago

Microsoft teams up with Cruise on development of self-driving cars

SPACS4 days ago

Take a Ride With Helicopter SPAC Experience Investment

Cyber Security3 days ago

Einstein Healthcare Network Announces August Breach