Multi-Cloud in Context: Comparing Cloud Survey Results

Multi-Cloud in Context: Comparing Cloud Survey Results

The inaugural HashiCorp State of Cloud Strategy Survey collected responses from more than 3,200 IT practitioners, technical decision makers, and business decision makers in HashiCorp’s opt-in database. That’s a lot of responses for a survey like this, and we’re excited about the insights revealed in the survey. But for added context, we compared five of the key results in our survey against other research to see where our conclusions were reinforced — and where they surfaced other perspectives.

»1. Multi-Cloud Is Here. And There. And Almost Everywhere.

Our survey made it clear that multi-cloud is no longer merely an aspirational goal but an everyday reality for the vast majority of respondents. More than three quarters of respondents said they were already using more than one cloud, and 86% said they expected to be doing so in two years.

Not so long ago, that result might have raised eyebrows, but now similar findings are showing up from many sources. In 2020, IDC predicted that by 2022, “over 90% of enterprises worldwide will be relying on a mix of on-premises/dedicated private clouds, multiple public clouds, and legacy platforms to meet their infrastructure needs.” Methodologies vary, but many recent surveys seem to indicate that’s exactly what’s happening. According to Flexera’s 2021 State of the Cloud Report, for example, 92% of enterprises have a multi-cloud strategy and 82% have a hybrid cloud strategy combining public and private clouds.

The just-released Equinix 2020-21 Global Tech Trends Survey

Largely concurs, indicating that from 2020 to 2012, “hybrid cloud adoption has increased by 12%, while multi-cloud models have increased 11%,” making hybrid cloud the most common deployment of cloud globally, with Asia-Pacific having the greatest level of penetration. By contrast, the Accelerate State of DevOps 2021 survey showed 34% using a combination of public cloud with private cloud/datacenter/on-premises deployments and 21% using multiple public clouds.

»2. Digital Transformation Isn’t Always the Top Driver of Multi-Cloud Adoption

In our survey, digital transformation was the top factor driving multi-cloud adoption, but it was ranked in the top three by only a third (34%) of respondents and followed closely by avoiding vendor lock-in (30%), cost reduction (28%), and scaling (25%).

What

The 2020 IDC report, meanwhile, suggested that multi-cloud helps enterprises achieve “better performance, 24/7 availability, enhanced security, and greater compliance with regulations.”

The Accelerate DevOps report found similar fragmentation. Leveraging the unique benefits of each provider was the most commonly named reason for using multiple cloud providers, but was chosen by only about a quarter (26%) of respondents (respondents were allowed to select only one). According to that report, this suggests that when respondents select an additional provider, they look for differentiation between their current provider and alternatives. The second most common reason for moving to multi-cloud was availability (22%). Unsurprisingly, respondents who have adopted multiple cloud providers were “1.5 times more likely to meet or exceed their reliability targets.”

»3. Cloud Investment Keeps Growing: Is it Paying Off?

Money always matters, and that’s certainly true in the cloud, where there’s increasingly big money at stake. According to the HashiCorp State of Cloud Strategy survey, more than 15% of respondents budgeted at least $10 million on their multi-cloud initiatives, and that figure ballooned to 34% of large enterprises. (Six percent of respondents budgeted $50 million or more.)

What

And in 2020, the IDC report suggested that respondents devote an average of $73.8 million — almost a third (32%) of their IT budget — to the cloud.

Of course, managing cloud spend is just as important as the actual numbers, and almost 40% of respondents to the HashiCorp survey ended up busting their cloud budget, spending more on the cloud than they planned. Surprisingly, bigger budgets seemed to lead to even more spending: 46% of companies budgeting $2 million to $10 million on cloud overspent, compared to just 27% of companies budgeting less than $100,000. Reasons for the overspending range from shifting priorities to COVID-19 to poorly managed resources.

Other surveys found similar results. In the Flexera survey, for example, 36% of enterprises said they spent more than $12 million on the cloud, while 83% spent more than $1.2 million. That’s up significantly from last year, when 20% of enterprises said they spent more than $12 million, and 74% spent more than $1.2 million.

The next question, obviously, is whether enterprises are getting the desired bang for their bucks, or is some of their cloud investment going to waste? According to the new PwC US Cloud Business Survey, “53% of companies have yet to realize substantial value from their cloud investments.” And a recent Virtana survey focusing on FinOps indicates that 82% of organizations have incurred “unnecessary” costs in their cloud operations due to workloads bursting above agreed capacity, overprovisioning of compute or storage resources, storage blocks that are no longer attached to a compute instance, poor job scheduling, over-buying, and other factors.

But the Accelerate DevOps Report offers a ray of hope in this area, noting that “respondents who use hybrid or multi-cloud were 1.6 times more likely to exceed their organizational performance targets.

»4. The Cloud Skills Shortage Is Worse Than You Think

One theme that emerged from the HashiCorp State of Cloud Strategy survey is that many organizations are struggling to develop the in-house skill sets they need to manage a robust cloud infrastructure. That was cited as a top-three multi-cloud challenge by 57% of respondents, and a top-three cloud inhibitor by 41% of respondents. The problem was especially acute in the public sector (53%) and consumer goods/retail companies (51%).

What

The need for “cloud native application development and operations skills” topped the list of 46% of hiring managers in a recent survey report from The Linux Foundation and edX. To be fair, though, the skills issue reaches beyond the need for cloud and container technologies, as ZDNet notes 92% of respondents said they were struggling to find new talent more generally and even to “hold onto existing talent in the face of fierce competition.”

The PwC Cloud Business Survey confirms that the shift to cloud has “only intensified” the “severe talent challenges that are a byproduct of digital transformation.” According to the report, more than half (52%) of “executives cite lack of tech talent — such as skills in cloud architecture, cybersecurity, or DevOps — as a barrier to realizing cloud value. And it gets worse: The report notes that “the digital talent divide affects not just tech specialists, but employees and business leaders who have the skills and mindset to thrive in a cloud-empowered world.”

»5. Yes, the COVID-19 Pandemic Is Accelerating Cloud Adoption

It’s tempting to blame companies scrambling to cope with the effects of the global pandemic for turbocharging the adoption of multi-cloud architectures. But while that’s clearly a factor, the situation is a bit more complicated. While more than half (54%) of HashiCorp survey respondents said that COVID had accelerated their cloud and multi-cloud adoption, most of them called the impact low or moderate.

How

Other surveys also showed pronounced pandemic effects, Almost half (47%) of global respondents to the Equinix survey said COVID had accelerated their digital transformation plans as a result of COVID-19. And 9 out of 10 respondents to the Flexera survey said COVID led to cloud usage “slightly higher than planned” (61%) or “significantly higher than planned” (29%).

But perhaps IDC put the COVID/cloud conundrum best by focusing not on specific effects, but on calling out the cloud’s ability to quickly adapt to changing conditions. As the research firm’s March 2020 press release (IDC Expects 2021 to Be the Year of Multi-Cloud as Global COVID-19 Pandemic Reaffirms Critical Need for Business Agility) noted, the story isn’t necessarily that the pandemic is driving specific technology solutions, but the fact that cloud is so well suited to helping enterprises cope with uncertainty and change. And that’s likely to remain valuable long after we put COVID-19 behind us.

»Learn More

For more insights into how companies are transitioning to the cloud and multi-cloud environments, and the benefits they’re getting from that move, check out the full HashiCorp State of Cloud Strategy Survey. And read more survey analysis on the HashiCorp blog.


Source: HashiCorp Blog

Announcing Terraform AWS Cloud Control Provider Tech Preview

Announcing Terraform AWS Cloud Control Provider Tech Preview

The HashiCorp Terraform AWS Cloud Control Provider, currently in tech preview, aims to bring Amazon Web Services (AWS) resources to Terraform users faster. The new provider is automatically generated, which means new features and services on AWS can be supported right away. The AWS Cloud Control provider supports hundreds of AWS resources, with more support being added as AWS service teams adopt the Cloud Control API standard.

For Terraform users managing infrastructure on AWS, we expect this new provider will be used alongside the existing AWS provider, which will continue to be maintained. Given the ability to automatically support new features and services, this new provider will increase the resource coverage and significantly reduce the time it takes to support new capabilities. We are excited for this to improve the experience and avoid the frustration caused by coverage gaps.

»AWS Cloud Control API

AWS Cloud Control API makes it easy for developers to manage their cloud infrastructure in a consistent manner and to leverage the latest AWS capabilities faster by providing a unified set of API actions as well as common input parameters and error types across AWS services. As a result, any resource type published to the CloudFormation Public Registry exposes a standard JSON schema and can be acted upon by this interface. AWS Cloud Control API is available in all commercial regions, except China.

For more information about AWS Cloud Control API, visit the user guide and documentation.

»How the AWS Cloud Control Terraform Works

Because AWS Cloud Control API provides an abstraction layer for resource providers to proxy through when interacting with AWS service APIs, we are able to automatically generate the codebase for the AWS Cloud Control Terraform provider. Generating the provider allows us to provide new resources faster because we won’t have to write boilerplate and standard resource implementations for each new service. The maintainers of the Terraform AWS Cloud Control provider can instead focus on user experience upgrades and performance improvements.

»Use Cases

While the Terraform AWS Cloud Control Provider is still in tech preview, we suggest practitioners use this provider to:

  • Experiment with new services before they are added to the Terraform AWS provider
  • Test configurations in development or staging environments
  • Build out proof-of-concept deployments in conjunction with the Terraform AWS provider, such as using Amazon AppFlow with Amazon S3 as illustrated in the example later in this blog post

Until the conclusion of the tech preview, we suggest using the Terraform AWS provider for production use across critical services. We will be evaluating the tech preview and will rely on community feedback to inform our decisions regarding general availability.

»Requirements

In order to use the new Terraform AWS Cloud Control provider, you will need:

  • Terraform 1.0. or later
  • An active AWS account in any commercial region, excluding China

»Configuring the Provider

In order to configure the provider, you will need to employ the configuration blocks shown here, while specifying your preferred region:

terraform {
  required_providers {
    awscc = {
      source  = "hashicorp/awscc"
      version = "~> 0.1"
    }
  }
}

# Configure the AWS Cloud Control Provider
provider "awscc" {
  region = "us-east-1"
}

»Authentication

To use the AWS Cloud Control provider, you will need to authenticate with your AWS account. You can use any authentication method available in the AWS SDK, including:

For more information and examples, please refer to the provider documentation on the Terraform Registry.

»Example Usage

To see how it all fits together, check out this example configuration using Amazon AppFlow. First, set up an AppFlow flow using the Terraform AWS Cloud Control API provider (awscc). Then set up an Amazon S3 bucket to store the flow data using the Terraform AWS provider (aws).

This example demonstrates how you can use the core resources in the aws provider to supplement the new services in the awscc provider.

»Using Two Providers

You’ll need to configure both providers in the same configuration file. Both the awscc and aws providers must be initialized for this example to work.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
    awscc = {
      source  = "something/awscc"
    }
  }
}
provider "aws" {
  region = "us-west-2"
}

provider "awscc" {
  region = "us-west-2"
}

»Setting Up S3 Buckets

Set up your S3 buckets using the aws provider. Designate a bucket for both your source and destination.

resource "aws_s3_bucket" "source" {
  bucket = "awscc-appflow-demo-source"
  acl    = "private"
}

resource "aws_s3_bucket" "destination" {
  bucket = "awscc-appflow-demo-destination"
  acl    = "private"
}

»Creating an AppFlow Flow

When creating a flow, you will need to provide the flow_name, connector_type, tasks, and trigger_config. Other optional attributes, such as tags can also be set on the resource.

To store flow data in S3, you must provide the bucket_name within the destination_connector_properties. You can also optionally provide the bucket_prefix and the s3_output_config.

resource "awscc_appflow_flow" "flow" {
  flow_name = "s3-to-s3-flow"
  source_flow_config = {
    connector_type = "S3"
    source_connector_properties = {
      s3 = {
        bucket_name = aws_s3_bucket.source.bucket
        bucket_prefix = "af"
      }
    }
  }
  destination_flow_config_list = [ 
    {
      connector_type = "S3"
      destination_connector_properties = {
        s3 = {
          bucket_name = aws_s3_bucket.destination.bucket
        }
      }
    }
  ]
  tasks = [
    {
        source_fields = [
            "column_one",
            "column_two"
        ]
        connector_operator = {
            s3 = "PROJECTION"
        }
        task_type = "Filter"
        task_properties = []
    },
    {
        source_fields = [
            "column_one,column_two"
        ]
        connector_operator = {
            s3 = "NO_OP"
        }
        destination_field = "column_cat"
        task_type = "Map",
        task_properties = [{
            key = "DESTINATION_DATA_TYPE"
            value = "string"
        }]
    },
    {
        source_fields = [
            "column_one",
            "column_two"
        ]
        connector_operator = {
            s3 = "NO_OP"
        }
        destination_field = "column_one,column_two"
        task_type = "Merge"
        task_properties = [{
            key = "CONCAT_FORMAT"
            value = "$${column_one} $${column_two}"
        }]
    }
  ]
  trigger_config = {
    trigger_type = "Scheduled"
    trigger_properties = {
        schedule_expression = "rate(1minutes)"
    }
  }
}

Note: At this time the AWS Cloud Control API does not offer the ability to schedule or start flows. To schedule your configured flow, you will need to use the AWS AppFlow console. For more information about how to use Amazon AppFlow and the various connection and destination types, visit the Amazon AppFlow documentation.

For additional examples visit the HashiCorp Learn Guide.

»Tell Us What You Think

We would love to hear your feedback on this project. You can report bugs and request features or enhancements for the AWS Cloud Control provider by opening an issue on our GitHub repository.

For AWS service coverage requests, please create an issue on the CloudFormation Public Coverage Roadmap.

For documentation and examples, please visit the learn Terraform Registry and HashiCorp Learn platform.


Source: HashiCorp Blog