Skip to main content

Network Layer

The Host is a WebSocket server that listens for incoming connections from Terminals.

The Terminal is a WebSocket client that connects to the Host.

This document describes how Hosts and Terminals establish connections and how Terminals exchange information with each other.

Hosts are typically accessible over the internet, while Terminals are usually located within local networks and are not accessible from the internet. Therefore, Terminals can connect to Hosts and transmit messages to each other through the Host.

Terminals are peer-to-peer and can be considered as a P2P network.

The Host is a dumb component that merely forwards messages between Terminals. Terminals, on the other hand, are intelligent components that can process messages.

Create Host​

You can use our provided host cluster service (wss://hosts.ntnl.io) or deploy your own host cluster (@yuants/app-hosts).

First, you need to generate an ED25519 key pair, then use the private key to sign an empty string to obtain the signature. Concatenate the public key and signature into the host connection URL to create a host URL.

The host URL looks like:

wss://hosts.ntnl.io/?public_key=84KD3d2vGLnYo5ars7eNG7KxZwWai2snuJBMFU8kbeg5&signature=3NP1aeyn88Lj7xJRoaDvzpfcLP8mMEp7CgMyXoQXS6MH5cpnpy62GkdeCkZhSwUdv4EJh8RX761e633cc5QtqeYw

Here, public_key is the host's public key; signature is the host's signature, which is the result of signing an empty string with the host's private key.

You need to use this URL to connect to the host. The host cluster, upon receiving this public key and signature, can verify if the signature matches the public key, thereby allowing you to connect to the host.

Terminals joining the host do not necessarily need to know the host's private key; they can obtain the host's connection URL from the host creator.

If the private key is accidentally leaked, the host creator can always create a new host key pair and migrate all services to the new host.

Connect to Host​

If you already have a host URL, you can connect to the host using the Web GUI, the Terminal class from @yuants/protocol, or any other WebSocket client.

When connecting to the host, the terminal needs to append its terminal ID to the host URL, like:

wss://hosts.ntnl.io/?public_key=84KD3d2vGLnYo5ars7eNG7KxZwWai2snuJBMFU8kbeg5&signature=3NP1aeyn88Lj7xJRoaDvzpfcLP8mMEp7CgMyXoQXS6MH5cpnpy62GkdeCkZhSwUdv4EJh8RX761e633cc5QtqeYw&terminal_id=haha

Here, terminal_id is the ID of the terminal. It should be unique within the host. This is similar to an IP address.

Security​

Before discussing security, it's important to clarify: Who are we? Who are we defending against?

In Yuan, all meaningful data is generated by terminals. We need to think about security from the terminal's perspective.

  1. Man-in-the-middle attacks. For example, could network providers, hackers, etc., eavesdrop on my messages?
  2. Software supply chain attacks. For example, could developers implant backdoors in the code?
  3. Attacks from the host cluster. For example, could hosts.ntnl.io listen to my messages?
  4. Attacks from the host creator. For example, could the host creator harm me?
  5. Attacks from other terminals in the host. For example, could other terminals in the host forge their identities and trick me into sending them private data?

How to Prevent Man-in-the-Middle Attacks​

Use TLS certificates to change the host URL to the wss:// protocol. Do not use the ws:// protocol.

For more information, refer to Wikipedia - Transport Layer Security.

How to Prevent Software Supply Chain Attacks​

Our code is open-source and supervised by the open-source community. You can participate in reviewing the source code. We will not do anything that harms users' interests in the code.

Additionally, you can check the hash value of the code package to ensure that the package you downloaded has not been tampered with. Modern npm package managers already support this feature and will automatically check the hash value during installation.

How to Prevent Attacks from the Host Cluster​

Attacks from the host cluster are essentially similar to man-in-the-middle attacks. Since the host cluster is the network provider, it has the ability to eavesdrop and tamper with messages passing through it.

However, since the terminals within the host are usually a group of people who know each other, they can use a pre-shared key method to ensure that the host cluster cannot eavesdrop on messages.

If the terminal knows the host's private key, this means the terminal is deployed by the host creator or has received full authorization from the creator. The terminal can directly derive an AES-GCM symmetric key from this private key and then encrypt the message before sending it. Since the host cluster does not know the private key, it cannot know the derived symmetric key and thus cannot decrypt the message.

If the terminal does not know the host's private key, it can request an additional pre-shared symmetric key from the host creator during deployment. This still ensures that the private key is not leaked, and the terminal can defend against attacks from the host cluster.

By the way, if the host cluster does not properly isolate communication between hosts, allowing another host to receive your messages or send messages to you, it is also equivalent to an attack from the host cluster. We can discuss one less scenario: attacks from other hosts.

How to Prevent Attacks from the Host Creator​

Good question, this is a difficult problem. But why would you take the risk of joining this host? It's your own choice.

Why not become a host creator yourself? You can create your own host. This way, you don't have to worry about attacks from the host creator.

How to Prevent Attacks from Other Terminals in the Host​

A stricter trust issue is that, assuming you join a host, you trust the host creator, but there are some other terminals in the host that you do not trust. You are worried that they might forge their identities and trick you into sending them private data.

The question now is, how to securely communicate with the host creator's terminal without allowing other participants to obtain private data?

Classically, we assume there are three terminals: Alice, Bob, and Eve.

You are Alice; Bob owns the host's private key; Eve is a malicious terminal.

  1. Eve has the ability to eavesdrop and tamper with messages within the host 100%, and even the host cluster is in cahoots with Eve, aiming to trick Alice into sending private data to herself.
  2. Bob owns the host's private key, follows the protocol, has no motive to leak Alice's private data, but does not know Alice beforehand and cannot inform Alice of any information in advance.
  3. Alice wants to send private data to Bob without sending it to Eve.

How to do it? Actually, the X25519 key exchange algorithm can solve this problem.

  1. Alice generates a one-time random X25519 key pair (Pub_A, Sec_A), sends Pub_A to the other terminal. The other terminal is also required to generate a one-time random X25519 key pair (Pub_B, Sec_B), then use the host's private key to sign "Pub_A+Pub_B". Then return the signature along with Pub_B.
  2. Alice can verify the signature to see if "Pub_A+Pub_B" matches the host's public key. If it does not match, it means the other terminal is not Bob, so regenerate the key pair and retry.
  3. Alice uses the obtained key Pub_B and her own Sec_A to calculate the shared key Shared_A_B, encrypts the private data with this key, and sends it to the other terminal.
  4. If the other terminal is Bob, he can use his Sec_B and Alice's Pub_A to calculate the same Shared_A_B, and use this key to decrypt the message.

What if the other terminal is actually Eve?

  1. If Eve gets "Pub_A", she cannot directly sign any message without the host's private key, so she cannot pass Alice's verification.
  2. Eve can only consider tricking Bob into signing a correct message. Suppose Bob's generated X25519 key pair is (Pub_B, Sec_B). The message that can be accepted by Alice is only in the format of "Pub_A+?". But the format of the signature returned by Bob must be "?+Pub_B". The intersection of the two formats has only one element: "Pub_A+Pub_B", so Eve cannot tamper with the message, Alice will definitely get Pub_B, and Bob will definitely get Pub_A. Of course, Eve will get Pub_A, Pub_B, but this is useless.
  3. According to the mathematical principles of the X25519 key exchange algorithm, Alice and Bob will definitely get the same Shared_A_B, while poor Eve cannot get this key, so she naturally cannot interpret the data encrypted with Shared_A_B.
  4. Eve can only rage impotently to destroy the network and hinder the normal communication between Alice and Bob, but she cannot steal private data.
What scenarios require such a complex encryption mechanism?

This is suitable for some visitor scenarios, for example, some terminals in the host are official, while others are visitors.

Visitor terminals can verify whether other terminals are official terminals through some methods.

Of course, visitor terminals can access all services within the host, so they still need to be cautious.

Summary​

Security and efficiency are mutually exclusive.

Stricter security means more verification computations, which will inevitably lead to a decrease in communication efficiency.

Improving efficiency means sacrificing some security, which also means saving money.

  1. If all your terminals are deployed within a LAN, you do not need to use TLS to defend against network man-in-the-middle attacks;
  2. If you have deployed your own host cluster or trust the host cluster provider, you do not need to guard against the host cluster stealing messages.
  3. If you join a host with strangers, you need to use the X25519 key exchange algorithm and add an additional per-message encryption mechanism to protect your privacy.

The best practice is that the division of privacy permissions is isomorphic to the division of hosts. The responsibilities of hosts are single.

  1. Deploy secrets known only to you to personal hosts that only you can access.
  2. Team-shared secrets can be deployed to public hosts that only team members can access.
  3. Use Portal terminals to authorize your services to other hosts for others to access.
  4. For providing services to strangers, create a new host, require strangers to use the key exchange algorithm to verify your identity, and then proceed with the service.
How to authorize across hosts?

A process can create multiple terminals to simultaneously connect to multiple hosts. This is very useful when you want to share data with others but do not want to disclose all your secrets. Hosts are very lightweight, and you can create a host at any time for a specific purpose. We will introduce separately how to use @yuants/app-portal to share information to a limited extent in the future.

Terminal Information​

Terminals should declare their Terminal information when connecting to a Host. Terminal information is a JSON object:

export interface ITerminalInfo {
  terminal_id: string;
  // Other fields omitted
}
  • terminal_id is the ID of the Terminal. It should be unique within the Host. This is similar to an IP address.
  • Terminal information usually includes service information of the Terminal. For example, a Terminal can declare that it provides account information services.
  • Terminal information is used for the Service Mode Layer, which we will introduce later.
  • The Host is responsible for broadcasting Terminal information to all Terminals in a timely manner.

Message Structure​

All messages are JSON-encoded and have the following structure:

export interface ITerminalMessage {
  source_terminal_id: string;
  target_terminal_id: string;

  // Other fields for the Service Mode Layer, to be introduced later.
}
  • source_terminal_id is the ID of the sender Terminal.
  • target_terminal_id is the ID of the recipient Terminal.
  • The Host reads the target_terminal_id and forwards the message to the target Terminal.
  • Other fields are used for the Service Mode Layer, which we will introduce later.

Optimization​

The following optimizations are not required by the protocol but implementing them can improve system performance. They are recommended.

Optimization: P2P Direct Connection​

We can use WebRTC to establish a P2P connection between two Terminals. WebRTC is a peer-to-peer technology that allows Terminals to exchange messages directly. It is a perfect choice for our purpose. Messages do not need to be transmitted through the Host. It is faster and incurs lower traffic costs. When a P2P connection is established, the Host will stop forwarding messages between the two Terminals. In fact, Terminals will no longer send messages to the Host. The operation of the Host has never changed. However, if the P2P connection is interrupted, the Host will resume forwarding messages between the two Terminals.

The Host will also forward ICE candidates (offers and answers) between the two Terminals. Therefore, the two Terminals can establish a P2P connection. The Host acts both as a STUN server and a TURN server.

Peer connections are established implicitly. When a Terminal sends a message to another Terminal, the Terminal will check if there is a P2P connection with the target Terminal. If not, the Terminal will attempt to establish a P2P connection with the target Terminal. If a P2P connection is established, the Terminal will send the message through the peer connection. If the P2P connection is interrupted, the Terminal will resume sending messages through the Host.

Optimization: Local Loopback​

If the target Terminal of a message points to the Terminal itself, then this message should not be sent over the network but directly to the Terminal's own message channel.

This is beneficial for Terminals to subscribe to channels provided by themselves or consume their own services, promoting design decoupling.