Skip to content

Blog

Why I have replaced Atlantis with BigConfig

Atlantis

As a long-time infrastructure enthusiast, I’ve had my share of dalliances with various tools and workflows. For a good while, Atlantis was my reliable partner in managing Terraform deployments. It brought order to the chaos of collaborative infrastructure-as-code, and for that, I’ll always be grateful.

However, like many relationships, sometimes you just grow apart. And in the rapidly evolving world of DevOps, staying stagnant means falling behind. So, after much deliberation, I’ve decided to move on from Atlantis for my Terraform needs, and I want to share why.

The Honeymoon Phase: What Atlantis Did Well

Section titled “The Honeymoon Phase: What Atlantis Did Well”

When Atlantis first arrived on the scene, it was a revelation. It solved a very real problem: how to bring a GitOps-like workflow to Terraform.

  • Pull Request Driven Workflow: This was Atlantis’s killer feature. The ability to run terraform plan and terraform apply directly from a pull request, with comments showing the output, was incredibly powerful. It made code reviews for infrastructure changes intuitive and collaborative.
  • Centralized State Management: By running within a controlled environment, Atlantis helped ensure that Terraform state was managed consistently and securely.
  • Concurrency Control: It prevented multiple users from running conflicting terraform apply commands on the same project, which was a lifesaver for team collaboration.
  • Simplicity of Setup (Initially): Getting Atlantis up and running wasn’t overly complex, especially for smaller teams.

For years, Atlantis was a solid choice, and it undoubtedly improved the way many teams managed their Terraform.

The Cracks Begin to Show: Where Atlantis Fell Short

Section titled “The Cracks Begin to Show: Where Atlantis Fell Short”

As my software engineering skills matured, some of Atlantis’s anti-patterns started to become more apparent.

  • Pull Request Driven Workflow: Atlantis, by design, has a somewhat opinionated workflow around the PR. But this is an anti-pattern if you follow David Farley’s principles of modern software engineering. The Trunk Based Development is incompatible with GitOps.
  • Hard to upgrade to an API or to a PaaS: While this is great for getting started, it becomes an obstacle if you want to upgrade your platform to an API or to a PaaS. The developer experience decreases because you end up with two solutions: one for K8s and for AWS/GCP/Azure resources.
  • The approval becomes becomes ineffective: The PR approach is an ineffective guard rail. Most of the reviews are not catching bugs before ending up in production.

The Modern Alternative: Why I’m Moving On

Section titled “The Modern Alternative: Why I’m Moving On”

So, how BigConfig is solving these limits of Atlantis?

  • Workflow client side: BigConfig has a cli that provides you with a workflow without a server side to coordinate changes made by multiple developers and to make sure that they are always applied sequentially.
  • It’s a library: The cli is also a library that can be used in any Clojure web service library like Pedestal to upgrade your infracoding to an API or to a PaaS.
  • Better developer experience: Let’s be honest, terraform is fondamentally an interactive tool. The scenario where a developer can write a non-trivial change in one go and then run it in Atlantis without bugs is rare. Most of the time we develop the terraform code incrementally with a sequence of plan and apply and the PR process increases the delivery time.

The Verdict: Atlantis is not compatible with modern software development

Section titled “The Verdict: Atlantis is not compatible with modern software development”

Let me be clear: Atlantis isn’t a “bad” tool. For many teams, especially those that are ok with the limits of the GitOps approach, it can still be a fantastic entry point. But for me, modern software development principles should be applied to both developement and operations.

In Berlin, where innovation moves at a rapid pace, staying agile means constantly re-evaluating your toolchain. My decision to move on from Atlantis is a reflection of that. I’m excited about the possibilities that BigConfig offers, allowing me to focus more on building reusable infrastructure.

Are you still using Atlantis? What are your thoughts? I’d love to hear your experiences.

Forward Deployed Engineering ❤️ Open Source

Merchant of complexity

Forward Deployed Engineering and Open Source as an alternative to Saas and Professional Services.

Section titled “Forward Deployed Engineering and Open Source as an alternative to Saas and Professional Services.”

In the evolving landscape of software, SaaS (Software as a Service) has long been the dominant model, offering convenience and accessibility. However, I expect a new paragigm to emerge: Forward Deployed Engineering for Open Source projects. This approach could challenge SaaS in the future by offering a compelling alternative that prioritizes control, customization, and community.

What is Forward Deployed Engineering declined for Open Source?

Section titled “What is Forward Deployed Engineering declined for Open Source?”

At its core, Forward Deployed Engineering declined for Open Source means that the engineers developing an Open Source project are actively involved in its deployment and operation within a user’s specific environment. Unlike SaaS, where the vendor manages everything, this model empowers users to run and even modify the software on their own infrastructure, with direct support and collaboration from the project’s core team.

Think of it as having the creators of the software as part of your extended team, helping you integrate, optimize, and troubleshoot it directly within your unique setup.

The SaaS Conundrum: When Convenience Comes at a Cost

Section titled “The SaaS Conundrum: When Convenience Comes at a Cost”

SaaS offers undeniable advantages: quick setup, automatic updates, and reduced operational overhead. But these benefits often come with significant trade-offs:

  • Vendor Lock-in: Migrating away from a SaaS provider can be notoriously difficult and costly, leading to reliance on a single vendor.
  • Limited Customization: SaaS solutions are designed for a broad audience, meaning deep customization to fit specific, niche requirements is often impossible or prohibitively expensive.
  • Data Control and Security Concerns: Users surrender control over their data to the SaaS provider, raising concerns about privacy, compliance, and security.
  • Opaque Costs: While seemingly straightforward, SaaS costs can escalate with usage, features, or user count, leading to unpredictable budgeting.
  • Lack of Transparency: The inner workings of a proprietary SaaS solution are a black box, making it difficult to diagnose issues or understand performance bottlenecks.
  • Merchant of complexity: Eventually all Saas becomes merchant of complexity.

How Forward Deployed Engineering Offers a Superior Alternative

Section titled “How Forward Deployed Engineering Offers a Superior Alternative”

For many organizations, particularly those with complex needs, strict security requirements, or a desire for ultimate control, Forward Deployed Engineering for Open Source projects offer a powerful alternative to SaaS.

  1. Ultimate Control and Ownership: With forward deployment engineering, you own the solution and the data. It runs on your infrastructure, giving you complete control over security, compliance, and data residency. There’s no vendor lock-in; you can switch providers or even manage it entirely in-house if you choose.

  2. Deep Customization and Flexibility: Open Source projects are inherently modifiable. Forward Deployed Engineering amplifies this by bringing the project’s experts directly to your environment. This allows for unparalleled customization, integration with existing systems, and the ability to tailor the software precisely to your unique workflows and requirements.

  3. Enhanced Security and Transparency: Running software on your own infrastructure with the help of the project’s engineers allows for greater control over security protocols. Furthermore, the Open Source nature means the code is auditable, providing transparency and reducing the risk of hidden vulnerabilities.

  4. Cost-Effectiveness at Scale: While initial setup might require more effort than a SaaS solution, the long-term costs can be significantly lower, especially at scale. You avoid recurring subscription fees that increase with usage, and you have the flexibility to optimize your infrastructure costs.

  5. Direct Collaboration and Community Benefits: Forward Deployed Engineering fosters a strong collaborative relationship between the user and the Open Source project team. This often leads to direct feature requests being implemented, bugs being squashed faster, and a stronger sense of community ownership over the software’s direction. Your operational challenges directly inform the project’s development.

When is Forward Deployed Engineering the Right Choice?

Section titled “When is Forward Deployed Engineering the Right Choice?”

Forward Deployed Engineering isn’t for everyone. Organizations that benefit most typically include:

  • Enterprises with complex IT landscapes: Those needing deep integration with existing systems.
  • Companies with strict regulatory or security requirements: Industries like finance, healthcare, or government.
  • Teams seeking maximum control and customization: When off-the-shelf solutions don’t quite fit.
  • Organizations with in-house technical talent: While the project engineers assist, some internal expertise is beneficial.

While SaaS will undoubtedly remain a popular choice for many, Forward Deployed Engineering for Open Source projects represents a powerful shift towards user empowerment and open collaboration. It’s not about outright replacing all SaaS, but rather providing a robust and often superior alternative for those who demand more control, transparency, and customization.

As Open Source projects continue to mature and offer enterprise-grade solutions, the model of Forward Deployed Engineering will likely become a cornerstone of how organizations adopt and leverage powerful software, shaping a future where the lines between vendor and user are increasingly blurred in favor of a collaborative, open ecosystem.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Merchant of complexity

Merchant of complexity

When Your Vendor Profits from Your Inefficiency

Section titled “When Your Vendor Profits from Your Inefficiency”

Have you ever felt like a vendor is subtly, or not so subtly, making things more complicated than they need to be? You’re not alone. This phenomenon, which I like to call the “Merchant of Complexity,” describes a business model where a vendor’s profitability is directly tied to the inefficiency of your internal processes. It’s a cunning, often insidious, way for them to extract more money from you over time.

The core strategy of a Merchant of Complexity is to introduce or maintain layers of intricacy that require their continued, and often expensive, intervention. Here’s how they typically operate:

  • Proprietary Systems and Black Boxes: They might offer solutions that are deliberately opaque or built on proprietary technology, making it difficult for your internal team to understand, manage, or modify. This creates a dependency on their experts for even minor adjustments or troubleshooting.
  • Perpetual Consulting and Support: Instead of providing a truly intuitive and self-sufficient product, they create a constant need for their consulting services, training, and ongoing support. Each new feature, update, or integration becomes an opportunity for them to bill for their time.
  • Fragmented Solutions: Rather than offering a comprehensive, integrated solution, they provide a series of disconnected modules or products. This forces you to spend more time and resources integrating these disparate parts, often with their assistance.
  • High Switching Costs: They might make it incredibly difficult to migrate away from their services. This could involve convoluted data export processes, non-standard data formats, or contract clauses that penalize early termination. Once you’re in, you’re stuck.

The immediate financial impact of dealing with a Merchant of Complexity is obvious: higher invoices for services, support, and customizations. However, the true costs run much deeper:

  • Decreased Productivity: Your team spends valuable time navigating unnecessarily complex systems, waiting for vendor support, or trying to piece together fragmented information.
  • Stifled Innovation: The difficulty in adapting or integrating new technologies can slow down your ability to innovate and respond to market changes.
  • Increased Frustration and Morale Issues: Employees become demoralized when faced with constant roadblocks and inefficiencies caused by vendor-imposed complexity.
  • Loss of Control: You cede control over your own processes and data, becoming reliant on an external entity whose interests may not align with yours.

Identifying and Combating the Merchant of Complexity

Section titled “Identifying and Combating the Merchant of Complexity”

So, how can you spot a Merchant of Complexity and protect your business?

  1. Question Complexity: If a solution seems overly complicated for the problem it’s solving, challenge it. Ask for simpler alternatives or explanations.
  2. Demand Transparency: Insist on clear documentation, open APIs, and transparent pricing structures. Avoid “black box” solutions where you don’t understand how things work.
  3. Prioritize Self-Sufficiency: Look for vendors who empower your team to be self-sufficient through good design, comprehensive training, and accessible resources.
  4. Evaluate Total Cost of Ownership (TCO): Don’t just look at the initial price tag. Consider the ongoing costs of support, customization, training, and potential inefficiencies.
  5. Seek Integrated Solutions: Whenever possible, opt for vendors who offer holistic, integrated solutions that minimize the need for complex integrations.
  6. Read Contracts Carefully: Pay close attention to clauses related to data ownership, migration, and termination. Understand the switching costs before you commit.

In today’s fast-paced business environment, efficiency is paramount. Don’t let a Merchant of Complexity hold your business hostage to unnecessary intricacy. By being vigilant and asking the right questions, you can avoid these pitfalls and choose partners who truly contribute to your success, rather than profiting from your struggles.

Are you facing simirar problems? What are your thoughts? I’d love to hear your experiences.