Announcing WunderHub: Share APIs like they were npm packages

Jens Neuse, CEO & Founder of WunderGraph

Do you remember how we shared applications before Docker existed? Applications were not as portable as they are today. You had to install all the required packages, manually or automated, but it was nowhere near as easy as docker run or docker build.

How portability and package managers changed the way we develop software#

Docker, or better yet OCI (Open Container Initiative), completely changed the way the bundle applications and distribute them. Today, you can pull an image from a Docker registry and run it on your local machine or in the cloud.

Similarly, could you imagine a world without package managers like npm, Composer, or Maven? There was a time when JavaScript libraries like jQuery had to be included directly into the HTML from a CDN. The way we develop software today heavily relies on package managers, bundlers, and build tools.

What both of these categories of tools have in common is that they fundamentally changed the way we develop software. They enabled new workflows and made it easier for developers to collaborate and share code.

Docker / OCI for example paved the way for kubernetes, standardizing the way how applications can be deployed in cloud native environments.

So what's the point of mentioning these two when we actually want to talk about APIs? Well, I believe that we're still in the stone age when it comes to sharing APIs.

While the API community developed tools like API Gateways and Developer Portals to secure and distribute APIs, they completely forgot to think about the developer experience of API developers and their consumers.

What happens when you visit a developer portal and decide, you'd like to use the API in your project? You download the SDK or use the Swagger / OpenAPI specification and start a manual integration process. There's no simple way to npm install the API and start using it.

The typical project doesn't just talk to a single database in isolation. You will probably have to integrate with multiple APIs from different teams or even third parties. Microservice architectures require a lot of integration work. Additionally, there are many powerful SaaS providers that offer APIs, e.g. for sending emails, managing user accounts, etc.

When it comes to integrating all these services, developers have to go through a lot of manual work. Wrapping SDKs, building backends for frontends, managing secrets and handling authentication are just a few of the problems to tackle. Most of the time, this manual integration work is not shared because it's proprietary closed source code that cannot be shared publicly. This means that API consumers do the same or similar work over and over again, wasting time and money.

We want to make API integration as simple as npm install. Our goal is to make APIs as portable as Docker containers, allowing API developers and their consumers to collaborate at a whole new level.

Manually integrating APIs is like importing jQuery from a CDN, let's get rid of that!

The solution: How to make APIs easily portable#

Similarly to Docker, we need a common language to make APIs portable. Additionally, we need a runtime to run our API integrations on.

Once we have these two, we need a place to store our API integrations so that API developers can "publish" their APIs and consumers can "pull" them into their projects, similar to Docker or npm.

GraphQL: The common language for API integrations#

For the language, we've decided to use GraphQL. By combining all APIs into a single GraphQL schema, we're able to "query" data from multiple APIs at once.

In that sense, we're creating a "Virtual Graph" that represents all API dependencies of a project. GraphQL was invented as a solution to fetch exactly the data needed by frontends.

GraphQL has a strong community and gets a lot of love from their users. Additionally, it comes with a powerful type system, making it very easy to generate e.g. TypeScript interfaces for your integrations.

WunderGraph: The runtime for API integrations#

What we've done for the last year is to build the runtime for API integrations. WunderGraph makes it easy to combine APIs from different services into a single GraphQL schema. Our Runtime / Engine is capable of combining them into a common format, allowing you to execute GraphQL Operations against almost any service.

So far, we're supporting the following Backends:

• REST (OpenAPI / Swagger)
• GraphQL
• Apollo Federation
• PostgreSQL
• MySQL
• SQLite
• SQL Server

You can "introspect" any of those and turn them into the "portable" WunderGraph format with a single command.

In addition to the above backends, we also support the following Frontends:

• REST(-ish) API
• Postman Collections
• Generated SDKs
  • TypeScript
  • React
  • React Native

When we talk about "frontends" we're talking about how to consume the API integrations. WunderGraph doesn't just combine your APIs into a GraphQL Schema and calls it a day. We go one step further, generating complete ready-to-use SDKs for your APIs that not just let you call your APIs, but also handle authentication and authorization, caching, security and more.

WunderHub: A place to store and share API integrations#

The last component of our solution is WunderHub. It's a place where you can store and share API integrations. Similarly to the Docker Hub or npm, you can publish your API descriptions and share them with the community.

You could share them publicly with everyone or limit access to just a group of people, e.g. only those of your own organization.

With the three components, the common language, the Runtime and the Hub, let's have a look at the flow of integrating APIs using WunderGraph and the Hub.

How does it work?#

Sharing an API using the WunderHub#

The first step is to introspect the API you want to share and translate it into the portable WunderGraph format. This can be done using the TypeScript SDK of WunderGraph. Here's an example:

// first, introspect an API
const countries = introspect.graphql({
    url: "https://countries.trevorblades.com/",
});

// then, prepare it for publishing it to the Hub
configurePublishWunderGraphAPI({
    organization: "wundergraph",
    apiName: "countries",
    title: "Trevor's Countries",
    markdownDescriptionFile: "countries.md",
    public: true,
    keywords: ["countries","trevor","trevorblades"],
    apis: [
        countries,
    ],
});

The SDK lets you introspect one or more APIs which you can then combine and publish. Similarly to how npm works, you publish the API to an organization and can use various methods to describe it.

Once you're ready to publish, run the following command:

$ wundergraph publish generated/wundergraph.countries.api.json

Integrating an API published on the Hub#

Now let's talk about the flow of integrating APIs using WunderGraph.

First, let's init a new project.

$ wundergraph init
$ cd .wundergraph
$ yarn

Then let's add two APIs to our workspace.

$ wunderctl integrations add wundergraph/countries wundergraph/spacex

The added API dependencies are automatically downloaded and installed. All API dependencies of a project are stored in the wundergraph.manifest.json file.

{
  "dependencies": [
    "wundergraph/spacex",
    "wundergraph/countries"
  ]
}

Once we've added the APIs to our workspace, we can add them to our WunderGraph API using the WunderGraph SDK.

import {
    Application,
    configureWunderGraphApplication,
    cors,
    templates
} from "@wundergraph/sdk";
import wunderGraphHooks from "./wundergraph.hooks";
import operations from "./wundergraph.operations";
import {integrations} from "./generated/wundergraph.integrations";

const spacex = integrations.wundergraph.spacex({
    apiNamespace: "spacex",
});

const countries = integrations.wundergraph.countries({
    apiNamespace: "countries",
});

const myApplication = new Application({
    name: "app",
    apis: [
        spacex,
        countries,
    ],
});

As you can see, we're instantiating both APIs from the generated "integrations" file. There's one little detail that might spark your attention, the apiNamespace parameter.

WunderGraph combines all your APIs into a single GraphQL Schema. If you combine APIs from different teams or vendors into the same GraphQL Schema, you're very likely to run into naming collisions and your Schema will be broken. By putting different APIs into their own namespace, we're able to avoid these problems without manual configuration.

As a final step, we have to define an operation to interact with our newly created API.

query DragonsAndContries {
    dragons: spacex_dragons {
        name
        active
    }
    countries: countries_countries {
        code
        name
        capital
    }
}

This Query retrieves data from both the SpaceX and Countries APIs. You can also see how the root level fields of both APIs are prefixed with the API namespace.

You're now ready to start your WunderGraph application and start using it.

# cd into the .wundergraph directory and run:
$ wunderctl up --debug

And finally, let's query it!

$ curl http://localhost:9991/api/main/operations/DragonsAndContries

In this example, we're simply using curl to query the generated REST(-ish) API, but you could also go more advanced and use a generated TypeScript client, the generated Postman Collection, etc...

Summary#

We've introspected a GraphQL API using the SDK, prepared it for publishing and then pushed it to the hub.

Then, as an API consumer, we've added two APIs to our project and instantiated them with an api namespace. Finally, we've defined an Operation and interacted with our newly created API integration using curl.

This might look like a simple example but I hope it's clear how much time we're able to save.

How does the world look like without using this flow?#

As we've said earlier, we think that API integrations are still in the stone age, so let's contrast the WunderGraph flow with how developers would save the same problem without WunderGraph.

• First, you have to decide on a technology, language and framework to build your REST API
• Then, add a new endpoint to your API
• Use a tool like graphql-code-generator to generate a typesafe API client for both APIs
• Use the generated client to query both APIs and implement the REST Endpoint
• Define a JSON Schema for the REST Endpoint
• Add an authentication and authorization layer to your REST Endpoint (this is batteries included in WunderGraph)
• Add a caching middleware (this is batteries included in WunderGraph)
• Use curl to query the REST Endpoint

We could easily make the list a lot longer because WunderGraph does way more than just integrating APIs. Have a look at our Features, our suite of tools helps you solve all problems around APIs, from authentication to authorization, role based access control, mocking, JSON Schema validation, automatic ETags, S3 file uploads and many more.

Additionally, imagine what happens if you have to add another API or one of the APIs needs to be updated. With WunderGraph and the Hub, it's a matter of minutes and mostly automatic. You shouldn't really waste your time for such a boring task.

Announcing the WunderHub closed Beta#

WunderGraph, the Runtime / Engine is very stable and ready for production. Together with our community of WunderGraph fans, we've been able to mature it over the last few months.

It's now time to move forward towards the final step before we want to release our framework to the public, Open Source and with no restrictions.

To make this final step easier, we need feedback from you, the community!

We're asking you to join our closed beta and help us to optimize the Developer Experience of both the WunderGraph Framework and the Hub.

If you're interested, please have a look at https://hub.wundergraph.com and sign up for the private beta. Additionally, you can also join our Discord and ask to join the Beta there.

We'd love to hear from you if you think the Hub and our Framework could help you to improve your experience of working with APIs.

The Future#

Our plan is to release our Open Source Framework in January 2022. Our long term goal is to be able to connect more backends and integrate with more frontend technologies.

On the backend side, we'd like to add support for SOAP, OData, gRPC and MongoDB. On the frontend side, we're looking at supporting Vue, Svelte, Angular, iOS, Android, Flutter, Java, Go, Python, Ruby, PHP, Typescript, Swift, Kotlin, C#.

Our vision is to become a meta-framework to solve all problems around APIs while being both backend and frontend agnostic. You should be able to use any backend or frontend technology, we're handling the heavy lifting of API integrations, security and more.

What to read next

This is a curated list of articles that I think you'll find interesting.

• In the WunderHub Announcement, I talk about how WunderHub will change the way we share and collaborate on APIs. It allows you to share APIs like npm packages.
• How automating API integrations benefits your business is dedicated to C-level executives who want to learn more about the business benefits of automating API integrations.
• Another interesting topic is to JOIN APIs without Schema Stitching or Federation, just by using a single GraphQL Operation
• For those interested in the most common GraphQL Security vulnerabilities, I suggest to read about them and how WunderGraph helps you to avoid them.
• A classic post but still relevant is I believe that GraphQL is not meant to be exposed over the Internet. It's a controversial topic and many misunderstand it. But think about it, why is HTTP not mentioned a single time in the GraphQL specification?
• One very common problem of using GraphQL is the Double Declaration Problem, the problem of declaring your types over and over again. This post explains that it's even more complicated than just double declaration and how we can solve it.
• The Fusion of GraphQL REST and HTTP/2 is a very long post, probably too long for a blog post. But if you're interested in a deep dive on the motivations behind creating WunderGraph, this is the post for you.

About the Author

Jens Neuse, CEO & Founder of WunderGraph

Jens has experience in building native apps for iOS and Android, built hybrid apps with Xamarin, React Native and Flutter, worked on backends using PHP, Java and Go. He's been in roles ranging from development to architecture and led smaller and larger engineering teams.

Throughout his whole career he realized that working with APIs is way too complicated, repetitive and needs a lot more standardization and automation. That's why he started WunderGraph, to make usage of APIs and collaboration through APIs easier.

He believes that businesses of the future will be built on top of collaborative systems that are connected through APIs. Making usage, exploration, sharing and collaboration with and through APIs easier is key to achieve this goal.

Follow and connect with Jens to exchange ideas or simply participate in his feed of thoughts.

Follow Jens on TwitterConnect with Jens on Linkedin

Comments

Discuss on TwitterDiscuss on HackerNewsDiscuss on RedditDiscuss on LinkedInDiscuss on Discord