Skip to main content

Programmatic deployment (v0.4.x)

· 4 min read

A new approach to deploying typegraphs has been introduced starting with version 0.4.0. This aims to facilitate the development of automation tools around the APIs you build within the Metatype ecosystem.

What has changed?

Before v0.4.x, we had to entirely rely on the meta cli to deploy typegraphs to a typegate instance.

This is no longer the case, as all core logic has been moved to the TypeScript/Python typegraph SDKs, both of which share the same WebAssembly-based typegraph-core behind the scenes. This provides some degree of assurance that you will have nearly identical experiences with each SDK.

What are the use-cases?

Since typegraphs can be written using the programming language your preferred SDK is based on, you can dynamically create typegraphs with ease.

The missing piece was having an interface natively backed inside the SDK for doing deployment programmatically.

Programmatic deployment

Initial setup

Just like any other dependency in your favorite programming language, each SDKs can be installed with your favorite package manager.

You can use one of the commands below to get started with the latest available version.

To upgrade the Typescript SDK of the typegraph package, you can use one of the following commands:

  • Node
npm update @typegraph/sdk
  • Deno
deno cache --reload "npm:@typegraph/sdk"

Configuration

This is analoguous to the yaml configuration file when you are using meta cli.

It's the place where you tell which typegate you want to deploy to, how you want the artifacts to be resolved, among other settings.

const config = {
typegate: {
url: "<TYPEGATE_URL>",
auth: new BasicAuth("<USERNAME>", "<PASSWORD>"),
},
typegraphPath: path.join(cwd, "path-to-typegraph.ts"),
prefix: "",
secrets: { POSTGRES: "<DB_URL>" },
migrationsDir: path.join("prisma-migrations", tg.name),
defaultMigrationAction: {
create: true,
reset: true, // allow destructive migrations
},
};

Deploy/remove

Now, picture this, you have a lot of typegraphs and one or more typegate instance(s) running, you can easily make small scripts that does any specific job you want.

// ..
import { tgDeploy, tgRemove } from "@typegraph/sdk/tg_deploy.js";
// ..

const BASIC_AUTH = loadMyAuthsFromSomeSource();
const TYPEGATE_URL = "...";

export async function getTypegraphs() {
// Suppose we have these typegraphs..
// Let's enumerate them like this to simplify
return [
{
tg: await import("path/to/shop-finances"),
location: "path/to/shop-finances.ts",
},
{
tg: await import("path/to/shop-stats"),
location: "path/to/shop-stats.ts",
},
];
}

export function getConfig(tgName: string, tgLocation: string) {
// Note: You can always develop various ways of constructing the configuration,
// like loading it from a file.
return {
typegate: {
url: "<TYPEGATE_URL>",
auth: new BasicAuth("<USERNAME>", "<PASSWORD>"),
},
typegraphPath: path.join(cwd, "path-to-typegraph.ts"),
prefix: "",
secrets: { POSTGRES: "<DB_URL>" },
migrationsDir: path.join("prisma-migrations", tg.name),
defaultMigrationAction: {
create: true,
reset: true, // allow destructive migrations
},
};
}

export async function deployAll() {
const typegraphs = await getTypegraphs();
for (const { tg, location } of typegraphs) {
try {
const config = getConfig(tg.name, location);
// use tgDeploy to deploy typegraphs, it will contain the response from typegate
const { typegate } = await tgDeploy(tg, config);
const selection = typegate?.data?.addTypegraph;
if (selection) {
const { messages } = selection;
console.log(messages.map(({ text }) => text).join("\n"));
} else {
throw new Error(JSON.stringify(typegate));
}
} catch (e) {
console.error("[!] Failed deploying", tg.name);
console.error(e);
}
}
}

export async function undeployAll() {
const typegraphs = await getTypegraphs();
for (const { tg } of typegraphs) {
try {
// use tgRemove to remove typegraphs
const { typegate } = await tgRemove("<TYPEGRAPH_NAME>", {
baseUrl: TYPEGATE_URL,
auth: BASIC_AUTH,
});
console.log(typegate);
} catch (e) {
console.error("Failed removing", tg.name);
console.error(e);
}
}
}

Going beyond

With these new additions, you can automate virtually anything programmatically on the typegraph side. Starting from having highly dynamic APIs to providing ways to deploy and configure them, you can even build a custom framework around the ecosystem!

Please tell us what you think and report any issues you found on Github.

Notes

You can check the Programmatic deployment reference page for more information.

The Node/Deno SDK is now available

· 2 min read

We are happy to announce that we have redesigned our SDKs to support Node/Deno and facilitate the integration of future languages. Most of the typegraph SDK is now written in Rust and shaped around a core interface running in WebAssembly.

Meet wit

In the realm of WebAssembly, the wit-bindgen project emerges as the most mature tool to create and maintain the language bindings for WebAssembly modules. This tool introduces WIT (WebAssembly Interface Types) as an Interface Definition Language (IDL) to describe the imports, exports, and capabilities of WebAssembly components seamlessly.

For example, Metatype implements the reactor pattern to handle requests as they come and delegate part of their execution in correct WASM runtime. The wit-bindgen helps there to define the interfaces between the guest (the Metatype runtime) and the host (the typegate) to ensure the correct serialization of the payloads. The wit definition could look like this:

package metatype:wit-wire;

interface typegate-wire {
hostcall: func(op-name: string, json: string) -> result<string, string>;
}

interface mat-wire {
record handle-req {
op-name: string,
in-json: string,
}

handle: func(req: handle-req) -> result<string, string>;
}

world wit-wire {
import typegate-wire;

export mat-wire;
}

The wit file is then used to generate the bindings for the host and the guest in Rust, TypeScript, Python, and other languages. The host bindings are used in the typegate to call the WASM runtime, and the guest bindings are used in the WASM runtime to call the typegate.

Install the v0.2.x series

The documentation contains now examples for Node and Deno.

Upgrade with Node

npm install @typegraph/sdk
meta new --template node .

Upgrade with Deno

meta new --template deno .
import { typegraph } from "npm:@typegraph/sdk/index.js";

Upgrade with Python

pip3 install --upgrade typegraph
poetry add typegraph@latest

Give us feedback!

This new release enables us to provide a consistent experience across all languages and reduce the work to maintain the existing Python SDK.

As always, report issues and let us know what you think on GitHub.

Programmable glue for developers

· 2 min read

We are introducing Metatype, a new project that allows developers to build modular and strongly typed APIs using typegraph as a programmable glue.

What is Metatype?

Metatype is an open source platform to author and deploy APIs for the cloud and components eras. It provides a declarative programming model that helps you to efficiently design APIs and focus on the functional requirements.

The runtime embraces WebAssembly (WASM) as a first-class citizen to allow you to write your business logic in the language of your choice and run it on-demand. Those "backend components" are reusable across your stacks and deployable without pipelines or containers.

The platform provides a set of capabilities out of the box:

  • create/read/update/delete data in your database
  • storing files in your cloud storage
  • authenticate users with different providers or using JWTs
  • connecting to third-party/internal APIs

And offers an opportunity to climb the one step higher in the abstraction ladder and drastically simplify the building of great APIs and systems!


Metatype is designed to be as simple as possible and horizontally scalable in existing container orchestration solution like Kubernetes. It consists of multiple parts, including:

  • Typegraph: a cross-language SDK to manage typegraphs - virtual graphs of types - and compose them
  • Typegate: a serverless GraphQL/REST gateway to execute queries over typegraphs
  • Meta CLI: a command-line tool to efficiently deploy the typegraphs on the gateway

What are virtual graphs?

Typegraphs are a declarative way to expose all APIs, storage and business logic of your stack as a single graph. They take inspiration from domain-driven design principles and in the idea that the relation between of the data is as important as data itself, even though they might be in different locations or shapes.

Loading...

These elements can then be combined and composed together similarly on how you would compose web components to create an interface in modern frontend practices. This allows developers to build modular and strongly typed APIs using typegraph as a programmable glue.

Where does this belong in the tech landscape?

Before Metatype, there was a gap in the technological landscape for a solution that specifically addressed the transactional, short-lived use cases. While there were existing tools for analytical or long-running use cases, such as Trino and Temporal, there was no generic engine for handling transactional, short-lived tasks.

← individual entities
transactional
large data →
analytical
instantaneous ↑
short-lived
Metatype
composition engine for entities in evolving systems
Trino
query engine for large data from multiples sources
long-running
asynchronous ↓
Temporal
workflow orchestration for long-running operations
Spark
batch/streaming engine for large data processing

Give it a try!

Let us know what you think! Metatype is open source and we welcome any feedback or contributions. The community primarily lives on GitHub.

Next steps

Emulating your server nodes locally

· 4 min read

Metatype is a platform which allows developers to solely focus on functional aspect of their applications by powering them with rich declarative API development tools to program and deploy in a cloud first environment. One component of Metatype is the Typegate, a serverless GraphQL/REST gateway for processing queries. This post is about how we in metatype made a dev friendly access to a typegate instance namely Embedded Typegate.

Introducing the Embedded Typegate

The embedded typegate is a feature that comes with the Meta CLI which provides the option of spinning a typegate instance from the CLI with minimum configurations and installations needed from the developer. All that is required to access the Embedded Typegate is to install Meta CLI. The spawned typegate instance behaves similarly to cloud-deployed typegates.

The motive

There are more than a couple of reasons why a developer would be tempted to use an emedded typegate. While developers can start a typegate instance using docker compose, the developer needs to install docker as a dependency to run the typegate container. Even though docker is familiar among many developers, it can sometimes be tricky and unbeknownst to some developers. We at metatype highly value the developer experience and one reason for adding the embedded typegate feature to the Meta CLI is for users to have a smooth experience with our system by providing a docker compose free experience. This feature provides a great utility for developers to author and test typegraphs in their local machine before deploying them to production level typegate instances on the cloud. Additionally, developers need not concern themselves with deployment configurations which are needed only during deployment. The only need to focus their energy and time in developing the right application and easily test them on embedded typegate running from the terminal. To add more to what is said, as the typegate engine keeps evolving, users will be abstracted away from the different configurations which might be added on the future. The Meta CLI will abstract much of what's not needed in a dev environment. Thus, leaving less headaches to developers on new changes. Ultimately, The embedded typegate is designed to be a good dev environment friendly tool which faciliates development time.

Quick First hand example

Install the v0.3.x series

Either of the two Typegraph SDKs are needed to author typegraphs. For this example, the node SDK will be used.

First, make sure the Meta CLI is installed.

curl -fsSL https://raw.githubusercontent.com/metatypedev/metatype/main/installer.sh | bash

Next, create a new node project using this command.

meta new --template node

The above command will create a sample typegraph which you can use to test the embedded typegate.

Now, you need to install the typegraph SDK by running the command down below. The previous command generates a package.json with the SDK specified as a dependency.

npm install

Before deploying the typegraph to the embedded typegate, Run the following commands below.

Execute this command to set tg_secret and tg_admin_password environment variables, which are neccessary for the typegate to run.

export tg_secret=a4lNi0PbEItlFZbus1oeH/+wyIxi9uH6TpL8AIqIaMBNvp7SESmuUBbfUwC0prxhGhZqHw8vMDYZAGMhSZ4fLw== tg_admin_password=password

Start the typegate instance.

meta typegate

Now that there is running instance of a typegate, you can deploy the example typegraph. From another terminal, run the command below.

meta deploy -f api/example.ts --allow-dirty --create-migration --target dev --gate http://localhost:7890

The typegate runs on port 7890 by default. If you access http://localhost:7890/example on your browser, you can see an GraphQL interface to interact with the deployed typegraph. You can test the example typegraph using the following graphql query below.

query {
multilpy(first: 3, second: 5)
}

Upgrade your Metatype development environment

To Upgrade the Meta CLI to the latest version, you can run the following command below.

meta upgrade

To upgrade the Typescript SDK of the typegraph package, you can use one of the following commands:

  • Node
npm update @typegraph/sdk
  • Deno
deno cache --reload "npm:@typegraph/sdk"

Learn more about Metatype

Wanna dive deep into the basics of Metaype? check our interactive tutorial revolving around the core features of the system.