Datacatalog Tag Template Processor

A package to manage Google Cloud Data Catalog Tag Template scripts.

Disclaimer: This is not an officially supported Google product.

Table of Contents

• Executing in Cloud Shell
• 1. Environment setup
  • 1.1. Python + virtualenv
    • 1.1.1. Install Python 3.6+
    • 1.1.2. Get the source code
    • 1.1.3. Create and activate an isolated Python environment
    • 1.1.4. Install the package
  • 1.2. Docker
  • 1.3. Auth credentials
    • 1.3.1. Create a service account and grant it below roles
    • 1.3.2. Download a JSON key and save it as
    • 1.3.3. Set the environment variables
• 2. Load Templates from CSV file
  • 2.1. Create a CSV file representing the Templates to be created
  • 2.2. Run the ddatacatalog-tag-template-processor script - Create the Tag Templates
  • 2.3. Run the datacatalog-tag-template-processor script - Delete the Tag Templates

---

Executing in Cloud Shell

# Set your SERVICE ACCOUNT, for instructions go to 1.3. Auth credentials
# This name is just a suggestion, feel free to name it following your naming conventions
export GOOGLE_APPLICATION_CREDENTIALS=~/datacatalog-tag-template-processor-sa.json

# Install datacatalog-tag-template-processor
pip3 install datacatalog-tag-template-processor --user

# Add to your PATH
export PATH=~/.local/bin:$PATH

# Look for available commands
datacatalog-tag-template-processor --help

1. Environment setup

1.1. Python + virtualenv

Using virtualenv is optional, but strongly recommended unless you use Docker.

1.1.1. Install Python 3.6+

1.1.2. Get the source code

git clone https://github.com/mesmacosta/datacatalog-tag-template-processor
cd ./datacatalog-tag-template-processor

All paths starting with ./ in the next steps are relative to the datacatalog-tag-template-processor folder.

1.1.3. Create and activate an isolated Python environment

pip install --upgrade virtualenv
python3 -m virtualenv --python python3 env
source ./env/bin/activate

1.1.4. Install the package

pip install --upgrade .

1.2. Docker

Docker may be used as an alternative to run the script. In this case, please disregard the Virtualenv setup instructions.

1.3. Auth credentials

1.3.1. Create a service account and grant it below roles

• Data Catalog Admin

1.3.2. Download a JSON key and save it as

This name is just a suggestion, feel free to name it following your naming conventions

• ./credentials/datacatalog-tag-template-processor-sa.json

1.3.3. Set the environment variables

This step may be skipped if you're using Docker.

export GOOGLE_APPLICATION_CREDENTIALS=~/credentials/datacatalog-tag-template-processor-sa.json

2. Load Templates from CSV file

2.1. Create a CSV file representing the Templates to be created

Templates are composed of as many lines as required to represent all of their fields. The columns are described as follows:

Column	Description	Mandatory
template_name	Resource name of the Tag Template for the Tag.	Y
display_name	Resource name of the Tag Template for the Tag.	Y
field_id	Id of the Tag Template field.	Y
field_display_name	Display name of the Tag Template field.	Y
field_type	Type of the Tag Template field.	Y
enum_values	Values for the Enum field.	N

2.2. Run the ddatacatalog-tag-template-processor script - Create the Tag Templates

• Python + virtualenv

datacatalog-tag-template-processor tag-templates create --csv-file CSV_FILE_PATH

2.3. Run the datacatalog-tag-template-processor script - Delete the Tag Templates

• Python + virtualenv

datacatalog-tag-template-processor tag-templates delete --csv-file CSV_FILE_PATH

TIPS

• sample-input/create-tag-templates for reference;