At Agira, Technology Simplified, Innovation Delivered, and Empowering Business is what we are passionate about. We always strive to build solutions that boost your productivity.

Cloud storage using StowUtils package in Golang

  • By Reddy Sai
  • February 23, 2017
  • 1792 Views

The intention of this article is to describe how to store files in cloud storage using the Golang stowutils package.

How does it work?

Implementations are referred to by a kind of string (like s3, google) and an object which provides specific configuration values associated with the implementation (like the S3 region, the credentials, the project ID in case of Google Cloud, etc.)
It uses stow package.
Stow provides implementations for cloud storage services. All providers almost have a similar approach when it comes to storing content files (or items) and are stored in buckets (or containers).
Object storage services are the concept around which the idea of stow is structured. It is made up of three main objects:

  • Location – The Container objects are stored here.
  • Container – Group of Item objects.
  • Item – an individual file.
location1 (e.g. s3)
container1
item1.1
item1.2
container2
item2.1
item2.2
location2 (e.g. google)
container1
item1.1
item1.2
container2
item2.1
item2.2

 
It should be noted that:

  • Location is like s3 or Google, and each location may have many containers (or storage buckets).
  • A container contains many items ( or files).
  • Containers do not contain other containers.
  • Items must belong to a container.

Pre-requisite:

Import Stowutils and implementations:
First, you import stowutils and any implementations you wish to support:

import (
utils "github.com/agiratech/stowutils"
)

 

Features of the Stowutils package:

  1. Get Location
  2. Acquire list of containers
  3. Get Container
  4. Acquire list of Items
  5. Get Item
  6. Upload Item
  7. Read Item
  8. Download Item
  9. Delete Item

Get Location

In order to get a location, you need to know the kind of string and a utils.Config object that contains the required configuration information (such as AccessKey, secretKey, JSON Credentials, projectID, etc). Configuration implementation is specific, and so, you should consult each implementation to see what fields are required for particular kind.
Example: For google cloud storage

g_location, err := utils.GetLocation("google",&utils.Config{
GoogleConfigJSONFilePath:"json-credentials.json",
GoogleProjectID:"sample-project"})
if err != nil {
return err
}
defer g_location.Close()
// use g_location

 
Example: For s3 cloud storage

s_location, err := utils.GetLocation("s3", &utils.Config{
S3AccessKeyID:"1452468515",
S3SecretKey:"a1b2c3d4",
S3Region:"us-west-2"})
if err != nil {
return err
}
defer s_location.Close()
// use s_location

 

Get list of containers

You can get Container list using the utils.ContainerList function:

func ContainerList(location stow.Location, prefix string, pageSize int) ([]stow.Container, error)

 
Here, prefix is a prefix name of containers.
Example:

g_containers, err := utils.ContainerList(g_location, "", 100)
if err != nil {
return err
}
for _,c := range g_containers {
// use container c
}

 

Get Container

You can get a required container using the utils.GetContainer function:

func GetContainer(location stow.Location, container_name string, pageSize int)
(stow.Container, error) {

 
Here, container_name is a required container name (and hence the container_name should be available)
Example:

g_container, err := utils.GetContainer(g_location, "sample-container", 100)
if err != nil {
return err
}
// use container

 

Get a list of Items

You can get item list using utils.ItemList function:

func ItemList(container stow.Container, prefix string, pageSize int) ([]stow.Item, error) {

 
Here, prefix is the prefix name of items.
Example:

g_items, err := utils.ItemList(g_container, "", 100)
if err != nil {
return err
}
for _,i := range g_items {
// use item i
}

 

Get Item

You can get a particular item using utils.GetItem function:

func GetItem(container stow.Container, item_name string, pageSize int) (stow.Item, error) {

 
The item_name mentioned here is a required item name (and hence the item_name should be available)
Example:

g_item, err := utils.GetItem(g_container, "img024.jpg", 100)
if err != nil {
return err
}
// use item

 

Upload Item

You can upload an item(file) using utils.UploadItem function:

func UploadItem(container stow.Container, file_path string, file_name string, metadata
map[string]interface{}) (stow.Item, error) {

 
Into whichever container you want to upload a file, pass that container as parameter. Here, file_path is an uploading file path and file_name is the name of the file, which should be unique. Otherwise, it will override an existing file.
Example:

item, err := utils.UploadItem(g_container, "/home/images/image001.jpg", "image001.jpg", nil)
if err != nil {
return err
}
// check the item

 

Read Item

You can read the item information using utils.ReadItem function:

func ReadItem(g_item stow.Item) (io.ReadCloser, error) {

 
Here, g_item is a received item using utils.GetItem
Example:

file, err = utils.ReadItem(item)
if err != nil {
return err
}
// use file

 

Download Item

Also, you can get the URL of the file using utils.DownloadItem function:

func DownloadItem(item stow.Item) *url.URL {

 

url = utils.DownloadItem(item)
// use URL

 

Delete Item

You can delete an item from a container using utils.DeleteItem function as mentioned below:

func DeleteItem(container stow.Container, item_name string) error {

 
Pass that container and file_name into the function as a params (the file_name should be available), which contains the file that you want to delete.
Example:

err = utils.DeleteItem(g_container, item.Name())
if err != nil {
return err
}

 

Conclusion

The above mentioned are all the features of the Stowutils package, and this article covers the basics of how it can be done, and what are the possibilities with it in Golang. Thus, the Stowutils package can be used to achieve the storing of files in the Cloud Storage. Try it out yourself for a better understanding.
 
We offer Golang development services for building world-class enterprise apps. We have expertise in building most complex software solutions using Google’s Go language. Chat with us now and hire golang developers within 72 hours.

Turn your vision to magnificent reality With
Our Web and Mobile Solutions