Building a Custom Terraform Provider
in Go — Deployed to TerraTowns

Problem

Terraform's power comes from its providers — the plugins that translate HCL resource definitions into real API calls. But when you need to manage infrastructure on a platform that doesn't have an existing provider, you hit a wall. The ExamPro Terraform Beginner Bootcamp 2023 used TerraTowns, a custom cloud hosting platform, as its deployment target. No public Terraform provider existed for it. Deploying homes to TerraTowns manually — outside of Terraform — would defeat the entire purpose of IaC: reproducibility, version control, and automated teardown. The challenge was to bring TerraTowns fully under Terraform's control, so that terraform apply could create, update, and destroy hosted content the same way it manages any AWS or GCP resource.

At the same time, each home needed real hosting infrastructure behind it — a place to serve static HTML and assets to the public. Standing that up manually for two separate homes would mean duplicated effort, inconsistent configuration, and no easy path to tearing it all down cleanly.

Solution

I built a custom Terraform provider in Go — terraform-provider-terratowns — that implements the full CRUD lifecycle for TerraTowns homes via its REST API. The provider is compiled into a local binary and resolved by Terraform through the local.providers/local/terratowns source path, making it a first-class citizen alongside any community provider.

For the hosting layer, I wrote a reusable Terraform module — terrahome_aws — that provisions an S3 bucket, CloudFront distribution, and Origin Access Control configuration as a single deployable unit. The module is instantiated twice: once for an Arcanum game guide, once for a Vietnamese Pho cooking page. The CloudFront domain name output from each module instance flows directly into the corresponding terratowns_home resource, wiring both layers together through Terraform's dependency graph. Remote state is managed in Terraform Cloud, and the entire dev environment is automated in Gitpod — so the full toolchain (Go, Terraform CLI, AWS CLI) spins up reproducibly from any fresh workspace.

Skills Acquired

• Terraform Provider Development (Go) — built a provider from scratch using the Terraform Plugin SDK, implementing resourceHouseCreate, resourceHouseRead, resourceHouseUpdate, and resourceHouseDelete as HTTP-backed CRUD functions. Understanding the provider contract — schema registration, d.SetId() semantics, and how Read syncs remote state back into the plan — is what separates provider users from provider builders.
• Terraform Modules — authored the terrahome_aws module with explicit input variables, locals, and three outputs (bucket_name, website_endpoint, domain_name). Modules enforce separation of concerns and make infrastructure reusable without duplication — the same principle as a well-designed function in application code.
• AWS S3 + Static Website Hosting — configured S3 for static asset delivery with index.html and error.html routing, uploaded files using fileset() for glob-based asset discovery, and managed upload triggers with a lifecycle block combining ignore_changes and replace_triggered_by.
• AWS CloudFront + Origin Access Control — provisioned a CloudFront distribution with OAC (SigV4 signing, signing_behavior = "always") and an S3 bucket policy that restricts access to a single distribution via AWS:SourceArn — the current best practice for secure S3-backed CDN architecture.
• Cache Invalidation via Provisioner — used a terraform_data resource with a local-exec provisioner to automatically run aws cloudfront create-invalidation --paths '/*' whenever content_version changes, ensuring stale content is never served after a deployment.
• Terraform Cloud (Remote State) — configured remote state in Terraform Cloud with state locking to prevent concurrent applies from corrupting the state file. Remote state is a prerequisite for any team-based Terraform workflow.
• Gitpod + Bash Scripting — automated the full toolchain setup through .gitpod.yml and scripts in bin/, using gp env to persist sensitive credentials (AWS keys, API tokens) across workspace restarts without committing them to version control.
• Go — used Go's standard net/http package for all provider HTTP calls, encoding/json for payload serialization, and the github.com/google/uuid library for UUID validation. Go's single static binary output is what makes it the language of choice for Terraform provider plugins.
• Ruby/Sinatra (Mock Server) — ran the terratowns_mock_server locally to test all four CRUD operations against a real HTTP server before touching the live platform, catching provider bugs without side effects.

Each skill reinforced a broader pattern: in infrastructure work, the integration between layers — provider to API, module to resource, CDN to bucket — is where complexity lives and where real understanding is built.

Deep Dive

Overview

This project came out of the ExamPro Terraform Beginner Bootcamp 2023 — a hands-on program that goes well beyond running terraform apply against existing providers. The core challenge: build a custom Terraform provider from scratch in Go that can create, read, update, and delete "homes" on TerraTowns, a real cloud hosting platform used as the bootcamp's deployment target.

Alongside the custom provider, I built a reusable Terraform module (terrahome_aws) that provisions the underlying AWS infrastructure — an S3 bucket for static asset storage and a CloudFront distribution for content delivery — and wired both together in a root main.tf that deploys two live homes: a guide on playing Arcanum in 2023 and a page about making Vietnamese Pho.

Architecture

The stack has two independent layers wired together through Terraform's dependency graph. The AWS layer (terrahome_aws module) provisions the hosting infrastructure. The TerraTowns layer (the custom provider) registers each home on the platform, using the CloudFront domain name output from the AWS module as its domain_name argument.

State is managed remotely in Terraform Cloud (organization testtf-organization, workspace terra-house-1), keeping the state file safe and enabling future team collaboration. The dev environment runs in Gitpod with automated install scripts so the full toolchain — Go, Terraform CLI, AWS CLI — spins up consistently from any fresh workspace.

The Custom Provider in Go

The terraform-provider-terratowns binary is compiled with go build into a local plugin path. It registers one resource type — terratowns_home — with five schema fields and implements all four CRUD functions using Go's standard net/http package. Three required provider inputs (endpoint, token, user_uuid) are passed into every HTTP request via a *Config struct.

func resourceHouseCreate(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
    config := m.(*Config)
    payload := map[string]interface{}{
        "name":            d.Get("name").(string),
        "description":     d.Get("description").(string),
        "domain_name":     d.Get("domain_name").(string),
        "town":            d.Get("town").(string),
        "content_version": d.Get("content_version").(int),
    }
    payloadBytes, _ := json.Marshal(payload)
    req, _ := http.NewRequest("POST",
        config.Endpoint+"/u/"+config.UserUuid+"/homes",
        bytes.NewBuffer(payloadBytes))
    req.Header.Set("Authorization", "Bearer "+config.Token)
    req.Header.Set("Content-Type", "application/json")
    // ... send, decode response, extract homeUUID ...
    d.SetId(homeUUID)
    return diags
}

if resp.StatusCode == http.StatusNotFound {
    d.SetId("")  // resource deleted outside Terraform
    return diags
}
// ... decode and set fields back into state ...

The terrahome_aws Module

The module at ./modules/terrahome_aws is instantiated twice — once per home — with different public_path and content_version inputs. It provisions seven resources and outputs bucket_name, website_endpoint, and domain_name (the CloudFront URL that gets passed to the custom provider).

resource "aws_s3_object" "upload_assets" {
  for_each = fileset("${var.public_path}/assets", "*.{jpg,png,gif}")
  bucket   = aws_s3_bucket.website_bucket.bucket
  key      = "assets/${each.key}"
  source   = "${var.public_path}/assets/${each.key}"
  etag     = filemd5("${var.public_path}/assets/${each.key}")
  lifecycle {
    replace_triggered_by = [terraform_data.content_version.output]
    ignore_changes       = [etag]
  }
}

resource "aws_cloudfront_origin_access_control" "default" {
  origin_access_control_origin_type = "s3"
  signing_behavior                  = "always"
  signing_protocol                  = "sigv4"
}

resource "terraform_data" "invalidate_cache" {
  triggers_replace = terraform_data.content_version.output
  provisioner "local-exec" {
    command = "aws cloudfront create-invalidation --distribution-id ${aws_cloudfront_distribution.s3_distribution.id} --paths '/*'"
  }
}

Root Configuration — Two Homes Deployed

The root main.tf instantiates the module and the custom provider resource twice. The CloudFront domain_name output from each module flows directly into its terratowns_home resource — Terraform resolves the dependency automatically.

module "terrahome_arcanum" {
  source          = "./modules/terrahome_aws"
  user_uuid       = var.user_uuid
  public_path     = var.arcanum.public_path
  content_version = var.arcanum.content_version
}

resource "terratowns_home" "arcanum" {
  name            = "How to play Arcanum in 2023!"
  description     = "Arcanum shipped with a lot of bugs but with mods it's really fun to play!"
  domain_name     = module.terrahome_arcanum.domain_name
  town            = "missingo"
  content_version = var.arcanum.content_version
}

module "terrahome_pho" {
  source          = "./modules/terrahome_aws"
  user_uuid       = var.user_uuid
  public_path     = var.pho.public_path
  content_version = var.pho.content_version
}

resource "terratowns_home" "pho" {
  name            = "It's Pho me!"
  description     = "How to make your own Pho at home!"
  domain_name     = module.terrahome_pho.domain_name
  town            = "missingo"
  content_version = var.pho.content_version
}

Reproducible Dev Environment — Gitpod

The full toolchain is automated through .gitpod.yml and bash scripts in bin/. Opening the repo in Gitpod triggers install scripts for the Terraform CLI, the AWS CLI, and Go. Gitpod's gp env command persists sensitive values — AWS credentials, the TerraTowns API token, user UUID — across workspace restarts without ever committing them to the repo.

Key Takeaways

• How Terraform providers work internally — implementing CRUD in Go made the provider contract concrete: schema registration, d.SetId("") for 404 handling, and why Read must sync state from the API rather than trusting local state.
• The lifecycle block is powerful — combining ignore_changes with replace_triggered_by on terraform_data gives fine-grained control over exactly when file uploads and cache invalidations fire.
• CloudFront + S3 with OAC — SigV4 signing with AWS:SourceArn in the bucket policy locks access to one specific distribution, not all of CloudFront.
• Go for infrastructure tooling — a single static binary, no runtime dependencies, straightforward once you understand the diag.Diagnostics return pattern.
• Remote state locks are non-negotiable — Terraform Cloud prevents concurrent applies from corrupting state, which is essential for any team environment.