Microservices in 2017 - Lessons from the Trenches
Note: This post is over 8 years old. The information may be outdated.
Microservices in 2017 - Lessons from the Trenches
July 2017 marks a pivotal moment in the evolution of microservices architecture. What began as an experimental approach championed by tech giants has now entered the mainstream, with organizations of all sizes decomposing monolithic applications into distributed services. As adoption grows, the community is developing more nuanced perspectives on when and how to implement microservices effectively.
The State of Microservices Adoption
The microservices landscape has matured significantly:
- Early Adopters: Companies like Netflix, Amazon, and Uber have been running microservices at scale for years
- Mainstream Adoption: Traditional enterprises across finance, retail, and healthcare are now embracing the approach
- Evolving Tooling: A rich ecosystem of tools has emerged to support microservices development and operations
This widespread adoption has generated valuable insights about what works and what doesn't in real-world implementations.
Key Implementation Patterns
Several patterns have emerged as particularly effective:
1. API Gateway Pattern
Providing a single entry point for clients while routing requests to appropriate services:
- Simplifying client interactions with the microservices ecosystem
- Handling cross-cutting concerns like authentication and rate limiting
- Enabling API composition for different client needs
Tools like Netflix Zuul, Kong, and AWS API Gateway have become standard components in microservices architectures.
2. Service Discovery
Dynamically locating service instances in highly dynamic environments:
- Client-side discovery using libraries like Netflix Eureka
- Server-side discovery using reverse proxies like NGINX
- DNS-based discovery for simpler implementations
3. Circuit Breaker Pattern
Preventing cascading failures across service dependencies:
- Failing fast when downstream services are unavailable
- Providing fallback mechanisms for degraded functionality
- Automatically recovering when services return to health
Netflix's Hystrix has become the reference implementation, though alternatives like Resilience4j are gaining traction.
4. Event-Driven Communication
Moving beyond simple REST APIs to more sophisticated communication patterns:
- Asynchronous messaging for loose coupling between services
- Event sourcing for maintaining complete audit trails
- Command Query Responsibility Segregation (CQRS) for separating read and write operations
Container Orchestration Comes of Age
2017 has seen container orchestration platforms mature into production-ready systems:
- Kubernetes: Emerging as the de facto standard for container orchestration
- Docker Swarm: Offering a simpler alternative with tight Docker integration
- Mesos/Marathon: Providing robust scheduling for diverse workloads
These platforms address critical operational challenges:
- Automated deployment and scaling
- Service discovery and load balancing
- Health monitoring and self-healing
Database Strategies for Microservices
Database architecture has emerged as one of the most challenging aspects of microservices:
1. Database per Service
The purist approach of giving each service its own database:
- Maximizing independence and autonomy
- Allowing each service to choose the right database technology
- Eliminating database-level coupling
2. Shared Database with Schema Separation
A pragmatic compromise for many organizations:
- Separate schemas or collections within a shared database
- Reducing operational complexity while maintaining some isolation
- Often used as a stepping stone toward full database separation
3. Data Replication and Synchronization
Addressing the challenge of maintaining data consistency:
- Change Data Capture (CDC) for replicating changes across databases
- Event sourcing for maintaining consistent event logs
- Eventual consistency models for scalable operations
Organizational Challenges and Conway's Law
The most significant challenges in microservices adoption often prove to be organizational rather than technical:
- Team Structure: Aligning team boundaries with service boundaries
- Ownership Models: Establishing clear ownership and accountability
- Cross-Cutting Concerns: Managing standards and shared capabilities
Organizations successfully implementing microservices are embracing DevOps culture and practices:
- Cross-functional teams with end-to-end responsibility
- Automated testing and deployment pipelines
- Shared on-call responsibilities across development and operations
Common Pitfalls and Anti-Patterns
As the community gains experience, several common mistakes have been identified:
1. Premature Decomposition
Breaking down systems into microservices before understanding domain boundaries:
- Creating services that are too fine-grained
- Introducing unnecessary network calls and latency
- Increasing overall system complexity without clear benefits
2. Distributed Monoliths
Creating ostensibly separate services that are tightly coupled:
- Synchronous chains of service calls
- Shared databases or schemas
- Lockstep deployment requirements
3. Neglecting Operational Complexity
Underestimating the operational challenges of distributed systems:
- Insufficient monitoring and observability
- Inadequate testing of failure scenarios
- Poor documentation of service interfaces and dependencies
Monitoring and Observability
The distributed nature of microservices demands sophisticated monitoring approaches:
- Distributed Tracing: Tools like Zipkin and Jaeger for tracking requests across services
- Aggregated Logging: Centralized logging with solutions like ELK stack
- Metrics Collection: Detailed performance metrics with Prometheus and Grafana
- Health Checking: Comprehensive health checks for automated recovery
Looking Ahead: The Future of Microservices
As we progress through 2017, several trends are emerging:
- Serverless Architectures: Functions-as-a-Service (FaaS) platforms taking microservices to their logical conclusion
- Service Mesh: Tools like Istio and Linkerd providing sophisticated traffic management
- Polyglot Persistence: Increasing specialization of database technologies for different service needs
- GraphQL: Providing more flexible APIs for frontend consumption of microservices
Conclusion: A Pragmatic Approach to Microservices
The most successful organizations in 2017 are taking a pragmatic approach to microservices:
- Starting with a monolith for new products and decomposing incrementally
- Focusing decomposition on business capabilities rather than technical layers
- Investing heavily in automation and operational tooling
- Building a culture that supports distributed system development
Microservices architecture offers powerful benefits in terms of scalability, resilience, and organizational alignment—but these benefits come with significant complexity costs. By learning from the experiences of early adopters and focusing on pragmatic implementation, organizations can navigate this complexity successfully.
As the microservices ecosystem continues to mature, we can expect further refinement of patterns, tools, and practices that make this architectural style more accessible to a broader range of organizations.
This article was written by Nguyen Tuan Si, a software architecture specialist with experience implementing microservices across various organization types and sizes.