NeuroAgent

Proper Dependency Injection Structure with Multiple Databases in Go

Optimal dependency injection architecture for working with PostgreSQL, MongoDB, and Redis in Go. Interface separation, lazy initialization, and best practices.

#database-design#dependency-injection-go#gin-framework#go-architecture#multiple-databases#postgres-mongo-redis#repository-pattern
Question

What should the proper dependency injection structure look like when working with multiple databases in Go?

Hello! I’m starting to learn server development in Go using the Gin framework. Currently, I’m setting up helper structures and dependencies. In my first project, I only used one database, but now I’m planning a large-scale project that will include PostgreSQL, MongoDB, and Redis.

The problem is as follows:

When working with a single database, my container looked like this:

go
type container struct {
    config *config.Config
    rep    repository.Repository
    logger logger.Logger
}

And the repository implemented methods for PostgreSQL:

go
type Repository interface {
    Model(value interface{}) *gorm.DB
    Select(query interface{}, args ...interface{}) *gorm.DB
    Find(out interface{}, where ...interface{}) *gorm.DB
    Exec(sql string, values ...interface{}) *gorm.DB
    First(out interface{}, where ...interface{}) *gorm.DB
    Raw(sql string, values ...interface{}) *gorm.DB
    Create(value interface{}) *gorm.DB
    Save(value interface{}) *gorm.DB
    Updates(value interface{}) *gorm.DB
    Delete(value interface{}) *gorm.DB
    Where(query interface{}, args ...interface{}) *gorm.DB
    Preload(column string, conditions ...interface{}) *gorm.DB
    Scopes(funcs ...func(*gorm.DB) *gorm.DB) *gorm.DB
    ScanRows(rows *sql.Rows, result interface{}) error
    Transaction(fc func(tx Repository) error) (err error)
    Close() error
    DropTableIfExists(value interface{}) error
    GetSQLDB() (*sql.DB, error)
    Clauses(conds ...clause.Expression) *gorm.DB
    AutoMigrate(value interface{}) error
}

type repository struct {
    db *gorm.DB
}

type filmRepository struct {
    *repository
}

If I put everything in one repository now, it would result in an overloaded interface. I came up with this implementation:

go
type Repository interface {
    Postgres() PostgresRepository
    Mongo() MongoRepository
    Redis() RedisRepository
}

That is, store three nested interfaces for each database type in the main interface. Is this a reasonable structure? I want to do everything correctly from the start rather than having to refactor later. If there are alternative solutions for such tasks, I’m ready to listen and take note.

NeuroAgent

Correct Dependency Injection Structure When Working with Multiple Databases in Go

The proper dependency injection structure when working with multiple databases in Go should ensure separation of concerns and facilitate easy testing. Based on your approach, I can suggest several improved solutions.

Main principle: Each database should have its own repository interface and be injected separately. This follows the Interface Segregation Principle.

Contents

Dependency Structure Approaches

1. Separate Interfaces for Each Database

This approach involves creating separate interfaces for each database and injecting them into the dependency container independently:

go
// Interfaces for each database
type PostgresRepository interface {
    FindUser(id uint) (*User, error)
    CreateUser(user *User) error
    // other PostgreSQL-specific methods
}

type MongoRepository interface {
    FindDocument(collection string, filter interface{}) (interface{}, error)
    InsertDocument(collection string, document interface{}) error
    // other MongoDB-specific methods
}

type RedisRepository interface {
    Set(key string, value interface{}, expiration time.Duration) error
    Get(key string) (string, error)
    // other Redis-specific methods
}

2. Your Current Approach with Nested Interfaces

Your approach with a single interface and Postgres(), Mongo(), Redis() methods is a facade, which can be useful but has drawbacks:

go
type Repository interface {
    Postgres() PostgresRepository
    Mongo() MongoRepository
    Redis() RedisRepository
}

Advantages:

  • Simplifies dependency passing to services
  • Provides a single point of access to repositories

Disadvantages:

  • Creates tight coupling between components
  • Complicates testing of individual repositories
  • May lead to excessive dependencies

The best practice is to use separate interfaces for each database with injection through a dependency container:

Dependency Container Structure

go
type container struct {
    config     *config.Config
    logger     logger.Logger
    postgresRepo PostgresRepository
    mongoRepo    MongoRepository
    redisRepo    RedisRepository
}

func NewContainer(cfg *config.Config, log logger.Logger) *container {
    return &container{
        config: cfg,
        logger: log,
    }
}

func (c *container) Postgres() PostgresRepository {
    if c.postgresRepo == nil {
        // Initialize PostgreSQL
        db, err := gorm.Open(postgres.Open(c.config.Database.URL), &gorm.Config{})
        if err != nil {
            c.logger.Fatal("Failed to connect to PostgreSQL", "error", err)
        }
        c.postgresRepo = &postgresRepository{db: db}
    }
    return c.postgresRepo
}

func (c *container) Mongo() MongoRepository {
    if c.mongoRepo == nil {
        // Initialize MongoDB
        client, err := mongo.Connect(context.Background(), options.Client().ApplyURI(c.config.MongoDB.URI))
        if err != nil {
            c.logger.Fatal("Failed to connect to MongoDB", "error", err)
        }
        c.mongoRepo = &mongoRepository{client: client}
    }
    return c.mongoRepo
}

func (c *container) Redis() RedisRepository {
    if c.redisRepo == nil {
        // Initialize Redis
        rdb := redis.NewClient(&redis.Options{
            Addr: c.config.Redis.Address,
        })
        c.redisRepo = &redisRepository{client: rdb}
    }
    return c.redisRepo
}

Repository Implementation Examples

PostgreSQL Repository:

go
type postgresRepository struct {
    db *gorm.DB
}

func (r *postgresRepository) FindUser(id uint) (*User, error) {
    var user User
    result := r.db.First(&user, id)
    if result.Error != nil {
        return nil, result.Error
    }
    return &user, nil
}

func (r *postgresRepository) CreateUser(user *User) error {
    return r.db.Create(user).Error
}

// Implementation of PostgresRepository interface

MongoDB Repository:

go
type mongoRepository struct {
    client *mongo.Client
}

func (r *mongoRepository) FindDocument(collection string, filter interface{}) (interface{}, error) {
    coll := r.client.Database("app").Collection(collection)
    var result bson.M
    err := coll.FindOne(context.Background(), filter).Decode(&result)
    if err != nil {
        return nil, err
    }
    return result, nil
}

func (r *mongoRepository) InsertDocument(collection string, document interface{}) error {
    coll := r.client.Database("app").Collection(collection)
    _, err := coll.InsertOne(context.Background(), document)
    return err
}

// Implementation of MongoRepository interface

Redis Repository:

go
type redisRepository struct {
    client *redis.Client
}

func (r *redisRepository) Set(key string, value interface{}, expiration time.Duration) error {
    return r.client.Set(context.Background(), key, value, expiration).Err()
}

func (r *redisRepository) Get(key string) (string, error) {
    return r.client.Get(context.Background(), key).Result()
}

// Implementation of RedisRepository interface

Alternative Solutions

1. Using Dependency Injection Frameworks

For more complex projects, you can use specialized libraries:

go
import "go.uber.org/dig"

func BuildContainer() *dig.Container {
    container := dig.New()
    
    container.Provide(func() (PostgresRepository, error) {
        // Initialize PostgreSQL
        db, err := gorm.Open(postgres.Open("..."), &gorm.Config{})
        return &postgresRepository{db: db}, err
    })
    
    container.Provide(func() (MongoRepository, error) {
        // Initialize MongoDB
        client, err := mongo.Connect(context.Background(), options.Client().ApplyURI("..."))
        return &mongoRepository{client: client}, err
    })
    
    container.Provide(func() (RedisRepository, error) {
        // Initialize Redis
        rdb := redis.NewClient(&redis.Options{Addr: "..."})
        return &redisRepository{client: rdb}, nil
    })
    
    return container
}

2. Service Layer with Aggregation

Create services that use multiple repositories:

go
type UserService struct {
    postgresRepo PostgresRepository
    mongoRepo    MongoRepository
    redisRepo    RedisRepository
}

func NewUserService(postgres PostgresRepository, mongo MongoRepository, redis RedisRepository) *UserService {
    return &UserService{
        postgresRepo: postgres,
        mongoRepo:    mongo,
        redisRepo:    redis,
    }
}

func (s *UserService) GetUserProfile(userID uint) (*UserProfile, error) {
    // Get data from PostgreSQL
    user, err := s.postgresRepo.FindUser(userID)
    if err != nil {
        return nil, err
    }
    
    // Get additional data from MongoDB
    profileData, err := s.mongoRepo.FindDocument("profiles", bson.M{"user_id": userID})
    if err != nil {
        return nil, err
    }
    
    // Cache in Redis
    s.redisRepo.Set(fmt.Sprintf("user:%d", userID), profileData, time.Hour)
    
    // Combine profile
    return &UserProfile{
        User:    user,
        Profile: profileData,
    }, nil
}

Practical Implementation of Dependency Container

Here’s a complete implementation of a dependency container with lazy initialization:

go
package di

import (
    "context"
    "log"
    "time"

    "github.com/go-redis/redis/v8"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
    "gorm.io/driver/postgres"
    "gorm.io/gorm"
)

type Container struct {
    config     *Config
    logger     Logger
    
    postgresRepo PostgresRepository
    mongoRepo    MongoRepository
    redisRepo    RedisRepository
}

type Config struct {
    Database DatabaseConfig
    MongoDB  MongoDBConfig
    Redis    RedisConfig
}

type DatabaseConfig struct {
    URL string
}

type MongoDBConfig struct {
    URI string
}

type RedisConfig struct {
    Address string
}

type Logger interface {
    Fatal(msg string, fields ...interface{})
}

type PostgresRepository interface {
    FindUser(id uint) (*User, error)
    CreateUser(user *User) error
    // Other PostgreSQL methods
}

type MongoRepository interface {
    FindDocument(collection string, filter interface{}) (interface{}, error)
    InsertDocument(collection string, document interface{}) error
    // Other MongoDB methods
}

type RedisRepository interface {
    Set(key string, value interface{}, expiration time.Duration) error
    Get(key string) (string, error)
    // Other Redis methods
}

func NewContainer(cfg *Config, log Logger) *Container {
    return &Container{
        config: cfg,
        logger: log,
    }
}

// PostgreSQL
func (c *Container) Postgres() PostgresRepository {
    if c.postgresRepo == nil {
        db, err := gorm.Open(postgres.Open(c.config.Database.URL), &gorm.Config{})
        if err != nil {
            c.logger.Fatal("Failed to connect to PostgreSQL", "error", err)
        }
        c.postgresRepo = &postgresRepository{db: db}
    }
    return c.postgresRepo
}

// MongoDB
func (c *Container) Mongo() MongoRepository {
    if c.mongoRepo == nil {
        client, err := mongo.Connect(context.Background(), options.Client().ApplyURI(c.config.MongoDB.URI))
        if err != nil {
            c.logger.Fatal("Failed to connect to MongoDB", "error", err)
        }
        c.mongoRepo = &mongoRepository{client: client}
    }
    return c.mongoRepo
}

// Redis
func (c *Container) Redis() RedisRepository {
    if c.redisRepo == nil {
        rdb := redis.NewClient(&redis.Options{
            Addr: c.config.Redis.Address,
        })
        c.redisRepo = &redisRepository{client: rdb}
    }
    return c.redisRepo
}

// Close all connections
func (c *Container) Close() error {
    var errors []error
    
    if c.redisRepo != nil {
        if err := c.redisRepo.(*redisRepository).client.Close(); err != nil {
            errors = append(errors, err)
        }
    }
    
    if c.mongoRepo != nil {
        if err := c.mongoRepo.(*mongoRepository).client.Disconnect(context.Background()); err != nil {
            errors = append(errors, err)
        }
    }
    
    if len(errors) > 0 {
        return errors[0] // Return first error
    }
    return nil
}

Code Organization Tips

1. Project Structure

/project
  /cmd
    /server
      main.go
  /internal
    /config
      config.go
    /di
      container.go
    /repositories
      /postgres
        repository.go
        user_repository.go
      /mongo
        repository.go
        document_repository.go
      /redis
        repository.go
        cache_repository.go
    /services
      user_service.go
      document_service.go
    /models
      user.go
      document.go
  /pkg
    /database
      postgres.go
      mongo.go
      redis.go

2. Proper Testing

Use mocks for testing services:

go
type MockPostgresRepository struct {
    users map[uint]*User
}

func (m *MockPostgresRepository) FindUser(id uint) (*User, error) {
    user, exists := m.users[id]
    if !exists {
        return nil, gorm.ErrRecordNotFound
    }
    return user, nil
}

func (m *MockPostgresRepository) CreateUser(user *User) error {
    m.users[user.ID] = user
    return nil
}

// Usage in test
func TestUserService_GetUser(t *testing.T) {
    mockPostgres := &MockPostgresRepository{
        users: map[uint]*User{
            1: {ID: 1, Name: "Test User"},
        },
    }
    
    userService := NewUserService(mockPostgres, nil, nil)
    
    user, err := userService.GetUser(1)
    assert.NoError(t, err)
    assert.Equal(t, "Test User", user.Name)
}

3. Configuration

Use structured configuration:

go
type Config struct {
    Server   ServerConfig   `yaml:"server"`
    Database DatabaseConfig `yaml:"database"`
    MongoDB  MongoDBConfig  `yaml:"mongodb"`
    Redis    RedisConfig    `yaml:"redis"`
}

type ServerConfig struct {
    Port int `yaml:"port"`
}

// Load configuration
func LoadConfig(path string) (*Config, error) {
    data, err := os.ReadFile(path)
    if err != nil {
        return nil, err
    }
    
    var config Config
    if err := yaml.Unmarshal(data, &config); err != nil {
        return nil, err
    }
    
    return &config, nil
}

Conclusions and Best Practices

Key Recommendations:

  1. Use separate interfaces for each database instead of one large interface
  2. Inject dependencies through a container with lazy initialization
  3. Follow the Interface Segregation Principle
  4. Test each repository separately using mocks
  5. Use a service layer for business logic that combines data from different sources

Why Your Current Approach May Be Suboptimal:

  • Tight Coupling: Services that only need Redis will be forced to depend on an interface that includes MongoDB and PostgreSQL
  • Complicated Testing: Testing services becomes more difficult due to the need to mock all three repositories
  • Scalability Issues: When adding a new database, you’ll need to modify the main interface

Alternative Approaches to Consider:

  1. Factory Approach: Create a factory for repositories
  2. Adapters: Use the adapter pattern to unify interfaces
  3. Contextual Dependencies: Pass required repositories through the HTTP request context

According to the Three Dots Labs blog, “The repository pattern works pretty nicely with Golang” when properly implemented with separate interfaces for different database types.

The correct dependency injection structure with multiple databases in Go should be flexible, testable, and easily scalable. Separating interfaces and using lazy initialization through a dependency container provides these qualities.

Sources

  1. Three Dots Labs - Repository pattern in Go
  2. Stack Overflow - How to handle DB connection in Go when using Repository pattern
  3. Golang Samples - PostgreSQL in Go: Repository pattern
  4. Medium - Repository Pattern in Golang
  5. Golang for All - Dependency injection in GO
How to properly test services using multiple repositories with different databases in Go?What are the advantages and disadvantages of using dependency injection frameworks (dig, wire) in Go for working with multiple databases?How to organize transactions when working with multiple databases in Go when data needs to be consistent?How to implement the Unit of Work pattern for working with PostgreSQL, MongoDB, and Redis in a single Go application?What are the best practices for managing connection pools when working with multiple databases in Go?How to properly handle errors when working with different types of databases in Go and how to unify them?
Ask NeuroAgent