Fast Data Transfer In and Out of Google Cloud Platform

Google Cloud is among the leading cloud platforms for enterprise organizations and SMBs, offering a wide range of compute and storage services. With powerful security and essentially unlimited scalability, many media organizations have turned to Google Cloud Platform as a key component in their architecture–especially in today’s hybrid cloud, multi-cloud world.

As businesses adopt Google Cloud Platform, they face critical decisions about the optimal transfer mechanism for moving their most vital content from on-premises into and out of Google Cloud.

Ingest presents a particularly significant challenge for data-intensive verticals such as media and entertainment. The sheer volume of data can be enormous, and it often arrives from multiple locations and is needed for time-critical operations.

In 2020, Signiant proudly announced support for Google Cloud Platform, providing fast, reliable, and secure solutions for accelerated data transfer into and out of the Google Cloud Platform from the Signiant platform.

Choosing the right approach to Google Cloud Storage data ingest
There are several options for moving data into Google Cloud Storage, including:

Shipping physical storage devices using the Google Transfer Appliance service;

• Private direct connectivity via Google’s
• Private Service Connect, Private Google Access, or Serverless VPC Access;
• Google Cloud migration utilities, including Storage Transfer Service, BigQuery Data Transfer Service or other applications built on Google Cloud’s SDKs;
• Signiant’s Flight, Manager+Agents, and Media Shuttle offerings.

Choosing the optimal solution for each use case depends on the amount of data, the frequency of data movement, the number of source locations, and the specific attributes of those locations.

Here are some things to consider when selecting the best tool for the job.

Physical Media: If you’re planning a one-time move of several petabytes of data and don’t have a tight timeline, shipment of physical storage devices can be a good choice. In-house resources at the data center are required to load the devices, and security concerns may prohibit physical shipments in some situations.

Private connectivity: If most of the data is in a single location and it’s moving to a single Google Cloud data center, a dedicated network pipe such as Private Service Connect can provide an efficient means of data transfer. The more widely dispersed the data, the more impractical it is to procure dedicated connectivity for all locations. Utilization is also an important consideration – depending on the data flow, you may end up paying for a connection that is often under-utilized.

Google Cloud Utilities: Many routine data transfer needs can be accommodated with tools like gcloud CLI. Because these utilities rely on TCP, however, they are susceptible to degraded performance in the presence of network congestion and/or when transmitting over longer distances (high latency). Therefore, it’s important to consider specific attributes of the source locations – including network capacity, network conditions, and distance from the destination Google Cloud data center.

Signiant acceleration for fast Google Cloud transfers

Signiant Flight with Google Cloud Storage: If you are experiencing unpredictable or slow performance with utilities like Google’s Storage Transfer Service, pairing the Storage Transfer Service software with Signiant Flight can solve the problem. This is particularly true if you’re transferring from multiple locations around the world and is invaluable if you plan to work with multiple cloud providers.

Flight can be easily deployed with any HTTP-based application and uses Signiant’s acceleration technology to minimize the impact of latency and congestion – thereby enabling full utilization of the available bandwidth. Because Flight can deploy anywhere, and you will only be charged when you’re transferring data, it can be thought of as a “Software-Defined Transfer Service.”

Signiant Flight with Signiant Manager+Agents: For advanced enterprise use cases, the combination of Manager+Agents and Flight provides a complete solution for orchestrating large-scale data movement in and out of Google Cloud Storage.

Data can be transferred over standard Internet connections or private Google Cloud Platform connections, with unified visibility and control of all transfers. This combination provides a highly reliable and configurable mechanism, as interrupted transfers automatically restart and Manager+Agents administrators can apply resource controls to the transfer.

Signiant Media Shuttle: Many cloud storage use cases involve data transfer by people, either as an ongoing part of the workflow or on an occasional, ad hoc basis. Signiant’s Media Shuttle SaaS offering meets this need via a simple web interface enabling larger file transfer to and from your Google Cloud Storage buckets. This makes it easy, fast and reliable for people to move any-size file into or out of Google Cloud and works across platforms so end-users have the same experience across on-prem storage or cloud storage from Google, Amazon Web Services or Microsoft Azure.

Benefits of Signiant for Google Cloud Storage data ingest

Signiant solutions provide unique value in situations where large volumes of data, long distances, multiple source locations, or flaky networks are involved. Unlike standard transfer mechanisms, our software makes the addition of Google Cloud Storage easy by providing:

Speed: With Signiant’s patented acceleration technology and scale-out architecture, you can take advantage of multi-Gbps transfer rates to Google Cloud Platform – even in the presence of latency and congestion.

Cloud-Native Implementation: Both Flight and Media Shuttle are cloud-native solutions, so there’s no need to procure or manage any cloud infrastructure. We handle all of that, and our 24/7 Site Reliability Engineering team monitors the system for thousands of customers worldwide.

Security: All Signiant solutions are designed from the ground up to be highly secure. Our software is trusted by most of the world’s top media enterprises to move their valuable content around the world.

Learn more about Signiant products for accelerated file transfers into and out of Google Cloud Platform.