Introducing Horizon 1.0

Horizon is an HTTP API server that makes it easy for people to interact with the Stellar network without having to worry about low-level details. It exposes multiple endpoints that are optimized to allow you to do things like:

• Submit a transaction to the network
• Read information about the current state of the ledger (like an account's balance)
• Explore historical data (like the transactions an account has submitted to the network)
  

If we were to take a picture of the Stellar universe, it would look like this:

Horizon sits between a product or service built on Stellar (wallet, asset issuer, payment provider) and a Stellar Core node.

That Stellar Core node is part of a peer-to-peer network: it coordinates with other nodes to validate transactions and maintain the current state of the ledger. It is not, however, optimized for developer interactions, especially for queries. The program it runs—also called Stellar-Core—encodes data in XDR (so it’s hard to read), and lacks useful indexes (so it’s hard to look things up). Unoptimized queries can actually slow a node down, which, worst-case, might lead it to lose track of consensus.

Horizon, on the other hand, is designed to allow developers to submit transactions and efficiently consume network data. It takes XDR data from Stellar-Core and decodes it into JSON, which is a lot easier to work with. Stellar SDKs are typically set up to poll Horizon endpoints, so projects building on Stellar aren’t constantly asking Stellar-Core questions. That means Stellar-Core can focus on the hard work of consensus and ledger ratification. Horizon, meanwhile, can focus on providing answers that are efficient and ergonomic.

Submitting a transaction to the network via Horizon is actually pretty straightforward: you send the transaction to Horizon and it proxies the transaction to a Stellar Core node.

Making it easy to read the current state of the ledger or exploring historical data requires more orchestration, and that's where Horizon offers the most value to its users. Through a process we call ingestion, Horizon takes data from the ledger and normalizes it so it’s easier to access and consume.

Back in August, we introduced a new system for handling ingestion, and published a blog post explaining the reasoning behind its creation. “It has some big advantages over the old ingestion system,” we wrote in that post, “it’s more consistent and developer-friendly, allows user configuration, and doesn’t overtax Stellar-Core—and eventually we plan to move over to it completely.”

The new ingestion system is a pretty big departure from the old one, and after months of refinement and road testing—first behind a feature flag, then in alpha and beta releases—it’s ready for production deployment. With Horizon 1.0, we switch off the old ingestion system and switch on the new one.

The new ingestion system introduces some breaking changes, which is part of why this is a major release. You can find the full list here. It also makes Horizon a lot nimbler. Here’s why:

With the old ingestion system, Horizon knew about the past, but not about the present. When you asked Horizon about historical transactions (payments, trades), it could find answers in its own database. If, however, you wanted to know something about the current state of the ledger (an account’s balance, open offers), Horizon didn’t know: it had to consult Stellar-Core’s database. The new ingestion system gives Horizon access to a full copy of the ledger state built using history archives and then dynamically updated, which means it can now see the present as well as the past. As a result, it’s more capable, more efficient, and less dependent on Stellar-Core

Switching Horizon’s guts means it can do more without bothering Stellar-Core. As a result, queries are faster and data is more consistent. Those two advantages are covered at length in our previous blog post, so we won’t really get into them here. The short version: longstanding 500 Internal Server Errors are a thing of the past, and path-finding is up to 10x faster. Also, for production cluster deployments, all instances can now ingest simultaneously, improving reliability through redundancy, and improving efficiency by spreading the ingestion work across the full cluster.

The new ingestion system also allows us to do more with Horizon, and this release includes some new endpoints that take advantage of that fact.

For instance: in the past, it was impossible to use Horizon to list all the accounts with a trustline to a given asset. For a lot of asset issuers that was a real problem: in the course of ordinary accounting, dividend payments, or KYC tracking, they needed to know how to identify relevant addresses, and there wasn’t an easy way to do that.

In Horizon 1.0, the /accounts endpoint includes a new query parameter ?asset which lists all the accounts with a trustline to a specified asset.

The request https://horizon.stellar.org/accounts?asset=NODE:GA7G44QLBUFKR36BWZN3ZC2IOFE4YAVU7DUOTFUKDDVMHG6VFA4SX4AJ returns all the accounts with a trustline to the asset identified by the code NODE, issued by the account GA7G44QLBUFKR36BWZN3ZC2IOFE4YAVU7DUOTFUKDDVMHG6VFA4SX4AJ.

Another example: in the past, it was impossible to use Horizon to list all the accounts a given public key can sign for. That makes things difficult if you have multiple multi-sig accounts: a business with several hot wallets and a cold wallet had to manually keep track of each address, and there was no definitive way to ensure they didn’t overlook one.

Now, the /accounts endpoint accepts a query parameter ?signer which lists all the accounts a given address can sign for.

The request https://horizon.stellar.org/accounts?signer=GD325AAA7IFJ6AW5G6PP5R36VL5NZFJOMJWKPK52RTECVJMODOTE2LX2 returns all the accounts with GD325AAA7IFJ6AW5G6PP5R36VL5NZFJOMJWKPK52RTECVJMODOTE2LX2 as a signer.

Finally, the new /offersendpoint allows you to find all offers on the network, and to filter results by seller, selling asset, or buying asset. If you want to see specific pairs for a given asset, or you want to take offers from a particular account, this one’s for you. You can use it to find out which accounts are trading on a given market, for instance, or bake it into a trading strategy that keeps closed loops of assets priced correctly.

The previous examples give a preview of the kind of things that are now possible thanks to the new ingestion system. Over subsequent releases, we plan to improve existing endpoints and unlock more possibilities to make Horizon serve you even better.

We’re planning to switch our public Horizon instances to 1.0 on March 2, 2020. You can track that upgrade and sign up for notifications on our statuspage. If you’re running your own Horizon, you can find all the details on how to migrate in our testing guide. Please make sure to take a look at the release notes for information about breaking changes, and to update your SDK so it’s compatible with the new version.

Working on this rewrite was our number one priority over the last couple of months. Now that it's finally out, we’re going to continue to improve and add features to Horizon, and to use the new ingestion system to explore new solutions to all kinds of problems. For example:

Currently, Horizon ingests all the activity on the network. If you’re an asset issuer, that may be overkill: you may only be concerned with your asset's activity, and may not want the burden of all that extra data. We’re starting to work on a plugin architecture so you can run Horizon keeping only the things that are relevant to you.

Until now Horizon has been in an unstable 0.x phase so breaking changes between versions were allowed. However, moving to 1.0 and a more mature deprecation scheme has been long overdue. Starting now, we will adhere to semantic versioning, giving you a better way to keep up with upcoming changes.

Last but not least, we want to take a moment to thank everyone who was involved in this release; we couldn't have done it without you.