Getting Started: Java

In this short tutorial, you'll learn how to create a server and client application using Java. We will create 3 different services: a ping service, a heartbeat service generating packets at regular intervals and a download service allowing you to download a file.

Creating the server

In order to create your server, create a gradle project in your favorite IDE and add the openzen repository and necessary artifacts:

repositories {
	maven { url "https://maven.openzen.org/" }
}

dependencies {
	compile 'org.openzen.packetstreams:PacketStreams:0.2.0'
	compile 'org.openzen.packetstreams:JavaCrypto:0.2.0'
}
	

The PacketStreams library contains both PacketStreams as well as its default communication protocol (QPSP). The JavaCrypto library provides support for running libsodium in desktop applications and will make sure the native libsodium library is correctly downloaded and installed. If you were developing a mobile application, you'd need a different Crypto library since the native library needs to be loaded differently.

In order to implement a server, you'll need a series of services. Although in future, certain services may be published as libraries, we will now write services ourselves.

Each service requires 2 classes to be implemented:

• A Service implementation, providing the server with your service metadata and a way to instantiate a connection.
• A ServiceStream implementation, handling the communication of a single service connection.

A Ping service

We'll start with a Ping service, which will retransmit incoming packets verbatim. This will be handy to test communication with our service. Let's start with the service class, which needs to implement the org.openzen.packetstreams.Service interface. This interface needs to implement 2 methods:

import org.openzen.packetstreams.Service;
import org.openzen.packetstreams.ServiceMeta;

class PingService implements Service {
	@Override
	public ServiceMeta getMeta() {
		// TODO
	}
	
	@Override
	public ServiceStream open(PacketStream stream, byte[] initializer) {
		// TODO
	}
}
	

The getMeta() method returns the service's metadata. A service's metadata will consist of the service UUID (identifying the protocol), priority, flags as well ass service-specific metadata. In our case, this is quite simple:

import java.util.UUID;
import org.openzen.packetstreams.Service;
import org.openzen.packetstreams.ServiceMeta;

class PingService implements Service {
	public static final UUID ID = UUID.parse("f0e5528d-8cee-4305-8f84-091f2cc92257");
	
	@Override
	public ServiceMeta getMeta() {
		return new ServiceMeta(
			ID,
			ServiceMeta.FLAG_META_CACHEABLE,
			0,
			new byte[0]);
	}
	
	@Override
	public ServiceStream open(PacketStream stream, byte[] initializer) {
		// TODO
	}
}

When implementing a standardized protocol (such as the ping protocol) you have to make sure to use the correct protocol UUID, in our case f0e5528d-8cee-4305-8f84-091f2cc92257. This is important to make sure that clients can verify that the service they are connecting with is indeed a ping service and not something else. When implementing your own service with a custom protocol, you'll need to generate a unique UUID yourself - for instance, by using online tools.

The ServiceMeta further specifies that the metadata can be cached client-side (making it faster to connect next time), that the priority is 0 (which is the default) and that the service info is an empty array, since the ping service protocol has no service metadata.

We now need to implement the actual protocol. To implement protocols, you can either implement org.openzen.packetstreams.ServiceStream directly, or create a subclass of org.openzen.packetstreams.BufferedServiceStream:

• ServiceStreams operate by informing the transmission system that new packets are available, and by implementing a next() method which returns a new packet, or null if no further packets are available.
• BufferedServiceStreams operate by having the implementation call send() with new packet data.

Underlying, BufferedServiceStreams implement ServiceStreams by keeping a buffer of data to transmit. Implementing your service using a BufferedServiceStream is generally easier if you are implementing an event-driven or request-response driven protocol; whereas implementing a ServiceStream directly is more nature when dealing with stream-oriented data. In the latter case, data can be loaded as the communication system requests it, automatically rate limiting outgoing data.

For the purpose of our ping protocol (which is request-response), we'll create a subclass of BufferedServiceStream:

import org.openzen.packetstreams.PacketStream;
import org.openzen.packetstreams.BufferedServiceStream;

class PingServiceStream extends BufferedServiceStream {
	public PingServiceStream(PacketStream stream) {
		super(stream);
	}
		
	@Override
	public void onConnected() {
		
	}

	@Override
	public void onReceived(byte[] packet) {
		send(packet);
	}

	@Override
	public void onConnectionClosed(int reason, byte[] info) {
		
	}
}

This will take any incoming packet and send it back to the client. We can then complete our Service class as well:

import java.util.UUID;
import org.openzen.packetstreams.Service;
import org.openzen.packetstreams.ServiceMeta;

class PingService implements Service {
	public static final UUID ID = UUID.parse("f0e5528d-8cee-4305-8f84-091f2cc92257");
	
	@Override
	public ServiceMeta getMeta() {
		return new ServiceMeta(
			ID,
			ServiceMeta.FLAG_META_CACHEABLE,
			0,
			new byte[0]);
	}
	
	@Override
	public ServiceStream open(PacketStream stream, byte[] initializer) {
		return new PingServiceStream(stream);
	}
}

This completes our service implementation.

Setting up the server

In order to make our service accessible to our client, we'll need a server to host this service. Currently the only available communication protocol is QPSP, so we can setup a QPSP endpoint server.

Setting up a server involves multiple steps:

• We need to provide the client with a way to validate that the server it's connecting with is indeed the correct one. This validation is performed using certificates. These certificates need to be signed by a root authority.
• We need to setup an endpoint with a host. This host can host one or more servers, each of which have a unique domain name. These servers then need to be able to resolve a service path to a service.

Let's create the server first, which is the easy part:

import org.openzen.packetstreams.BaseServer;
import org.openzen.packetstreams.Service;
import org.openzen.packetstreams.crypto.CertificateChain;
import org.openzen.packetstreams.crypto.CryptoKeyPair;

public class TestServer extends BaseServer {
	private final Service ping = new PingService();
	
	public TestServer(CryptoKeyPair keyPair, CertificateChain certificate) {
		super(keyPair, certificate);
	}

	@Override
	public Service getService(String path) {
		switch (path) {
			case "ping":
				return ping;
			default:
				return null;
		}
	}
}

Servers need a keypair and a certificate (which we will generate below), and will need a method to resolve paths to their respective service. By extending BaseService, all we have to do is implement the getService method, resolving pathnames to their respective service. In our example, we'll have our ping service at the path named "ping". This is implemented with a simple switch but more advanced applications may provide their own routing logic here. Note that returning null from this method means that there is no service at the given path.

We can then carry on and take care of setting up a secure server. For the purpose of our application, we'll automatically generate our own root authority and save its information to a file:

package org.openzen.packetstreams.testserver;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.util.Base64;
import org.openzen.packetstreams.crypto.CryptoProvider;
import org.openzen.packetstreams.crypto.CryptoSigningKey;
import org.openzen.packetstreams.javacrypto.JavaCrypto;

public class Main {
    public static void main(String[] args) throws IOException {
		CryptoProvider crypto = JavaCrypto.INSTANCE;
		
		File signingKeyFile = new File("SigningKey.dat");
		CryptoSigningKey signingKey;
		if (!signingKeyFile.exists()) {
			signingKey = crypto.generateSigningKey();
			Files.write(signingKeyFile.toPath(), signingKey.encode());
			byte[] signingKeyAsString = Base64.getEncoder().encode(signingKey.getVerifyKey().encode());
			Files.write(new File("VerifyKey.txt").toPath(), signingKeyAsString);
		} else {
			signingKey = crypto.decodeSigningKey(Files.readAllBytes(new File("SigningKey.dat").toPath()));
		}
	}
}
	

Running this application for the first time will generate 2 files, SigningKey.dat and VerifyKey.txt. SigningKey.dat contains the private signing key of your authority and should be kept safe. VerifyKey.txt contains the public verify key in base64 format and can be distributed with your client application. Whenever you connect with your server, you must validate that the root is indeed the one used on your server, or a third party might be eavesdropping your communication!

Next time you run, the server will pick up these files, making sure the signing and verify keys don't change with every run.

We can then generate a keypair and its associated certificate:

/* ... snip ... */
import org.openzen.packetstreams.crypto.CertificateChain;
import org.openzen.packetstreams.crypto.CryptoKeyPair;

public class Main {
	public static void main(String[] args) throws IOException {
		/* ... snip ... */
		
		String domainName = "localhost";
		CryptoKeyPair keyPair = crypto.generateKeyPair();
		CertificateChain certificate = new CertificateChain(signingKey, keyPair.publicKey, domainName);
	}
}

The code above will use "localhost" as domain name for local testing, but you can specify any domain name you wish (or an IP address).

This allows us to create our server and setup a server endpoint:

/* ... snip ... */
import org.openzen.packetstreams.SingleServerHost;
import org.openzen.packetstreams.qpsp.QPSPEndpoint;

public class Main {
	public static void main(String[] args) throws IOException {
		/* ... snip ... */
		
		TestServer server = new TestServer(keyPair, certificate);
		SingleServerHost host = new SingleServerHost(domainName, server);
		QPSPEndpoint endpoint = new QPSPEndpoint(host, crypto);
		endpoint.open();
	}
}

This will setup a host with a single server (at a single domain) at the default port 1200. Our server is ready!

Setting up the client

The client will now need to connect with the server, connect to its ping service and transmit packets through it.

In order for a client to connect to a service, it will need to implement 2 classes as well:

• A service connector which receives the metadata of the service it is connecting to, which will need to generate the correct initialization data and open a client-side service stream when the connection is opened.
• A client-side service stream implementation.

For the ping service, not much needs to happen:

public class PingConnector implements ServiceConnector {
	public static final UUID ID = UUID.parse("f0e5528d-8cee-4305-8f84-091f2cc92257");
	
	@Override
	public byte[] connect(ServiceMeta meta) {
		if (!meta.uuid.equals(ID))
			return null;
		
		return new byte[0];
	}

	@Override
	public byte[] connectWithUpdatedMeta(ServiceMeta meta) {
		return connect(meta);
	}

	@Override
	public ServiceStream onConnected(PacketStream stream) {
		return new PingClientStream(stream);
	}
}

The connect and connectWithUpdatedMeta methods will be informed about the metadata of the service it is trying to connect with. The connector can then decide to proceed with connection, returning connection initialization data. This initialization data will be transmitted to the service (an example of which will be illustrated later). A connector may also decide to reject the connection by returning null.

In our example, we'll verify that the service we are connecting to is indeed the ping service and if that's indeed the case, give it an empty byte array since the ping service doesn't need initialization data.

Note the connectWithUpdatedMeta method, which is important when dealing with cached service metadata. Generally you'll want to forward this call to the connect method, but more advanced applications may need a custom implementation.

We can then implement the actual client stream:

class PingClientStream extends BufferedServiceStream {
	public PingClientStream(PacketStream stream) {
		super(stream);
	}
	
	@Override
	public void onConnected() {
		for (int i = 0; i < 10; i++)
			send(i, "Test message");
	}
	
	@Override
	public void onReceived(byte[] packet) {
		BytesDataInput input = new BytesDataInput(packet);
		int index = input.readVarUInt();
		System.out.println("Client received " + index + ": " + input.readString());
	}

	@Override
	public void onConnectionClosed(int reason, byte[] info) {
		
	}
	
	private void send(int index, String message) {
		BytesDataOutput output = new BytesDataOutput();
		output.writeVarUInt(index);
		output.writeString(message);
		send(output.toByteArray());
	}
}

This client will, upon connection, transmit 10 messages. Each message will contain an index as well as a string, which are combined in a packet. This example makes use of BytesDataOutput and BytesDataInput helper classes provided (and internally used) by PacketStreams for compact data encoding.

Now that we have a client connector and stream, we can perform the actual connection initialization:

class MainClient {
	public static void main(String[] args) throws IOException {
		CryptoProvider crypto = JavaCrypto.INSTANCE;
		QPSPEndpoint client = new QPSPEndpoint(0, crypto);
		
		byte[] rootKey = Base64.decode("{SigningKey.txt}", Base64.DEFAULT);
		PinnedRootValidator rootValidator = new PinnedRootValidator(rootKey);
		ClientSession clientSession = client.connect("localhost", rootValidator);
		clientSession.whenEstablished(session -> session.open("ping", new PingConnector()));
		client.open();
	}
}

This will connect to the localhost server and open the ping server. Our client is ready!

Note that since service references require a server and path, they can be written als URLs. The standard format of an URL accessible through qpsp is qpsp://{host}/{path}. For instance, our ping service resides at qpsp://localhost/ping. Note that paths do not contain the initial slash! (and qpsp://localhost and qpsp://localhost/ are equivalent).

A heartbeat service

We can then continue to expand our server which a second service, which transmits packets at regular intervals. We let the client choose the interval upon connection.

Let's start by implementing a new service:

import java.util.UUID;
import org.openzen.packetstreams.PacketStream;
import org.openzen.packetstreams.Service;
import org.openzen.packetstreams.ServiceMeta;
import org.openzen.packetstreams.ServiceStream;
import org.openzen.packetstreams.io.BytesDataInput;

public class HeartbeatService implements Service {
	public static final UUID ID = UUID.fromString("b4c1683e-1421-478b-90a3-ead8b50547bd");
	private static final ServiceMeta META = new ServiceMeta(ID, ServiceMeta.FLAG_META_CACHEABLE, 0, new byte[0]);
	
	@Override
	public ServiceMeta getMeta() {
		return META;
	}

	@Override
	public ServiceStream open(PacketStream stream, byte[] initializer) {
		BytesDataInput input = new BytesDataInput(initializer);
		int intervalMillis = input.readVarUInt();
		return new HeartbeatServiceStream(stream, intervalMillis);
	}
}

This service implementation determines the service's UUID, default priority, no service metadata. This time, we're letting the client specify the heartbeat's interval, and we specify this using initialization data. Initialization data is transmitted when a connection is started and is comparable to having HTTP headers in an HTTP request. PacketStreams makes no assumption whatsoever about the initialization data, it's entirely up to the service protocol.

We can then implement the actual service:

import java.util.Timer;
import java.util.TimerTask;
import org.openzen.packetstreams.BufferedServiceStream;
import org.openzen.packetstreams.PacketStream;
import org.openzen.packetstreams.io.BytesDataOutput;

public class HeartbeatServiceStream extends BufferedServiceStream {
	private final Timer timer = new Timer();
	private final int intervalMillis;
	
	public HeartbeatServiceStream(PacketStream target, int intervalMillis) {
		super(target);
		
		this.intervalMillis = intervalMillis;
	}

	@Override
	public void onConnected() {
		timer.scheduleAtFixedRate(new TimerTask() {
			@Override
			public void run() {
				BytesDataOutput output = new BytesDataOutput();
				output.writeVarLong(System.currentTimeMillis());
				send(output.toByteArray());
			}
		}, intervalMillis, intervalMillis);
	}

	@Override
	public void onReceived(byte[] packet) {
		
	}

	@Override
	public void onConnectionClosed(int reason, byte[] info) {
		timer.cancel();
	}
}

Upon connection, the service will schedule a timer at regular intervals as determined by the client. We're using BufferedServiceStream again, since we have an event-oriented stream. Note that the send method in BufferedServiceStream is thread-safe, so you can call it from any thread. When the connection is closed, we cancel the time and that's pretty much it.

In order to make the service accessible, let's expand the Server class as well:

import org.openzen.packetstreams.BaseServer;
import org.openzen.packetstreams.Service;
import org.openzen.packetstreams.crypto.CertificateChain;
import org.openzen.packetstreams.crypto.CryptoKeyPair;

public class TestServer extends BaseServer {
	private final Service ping = new PingService();
	private final Service heartbeat = new HeartbeatService();
	
	public TestServer(CryptoKeyPair keyPair, CertificateChain certificate) {
		super(keyPair, certificate);
	}

	@Override
	public Service getService(String path) {
		switch (path) {
			case "ping":
				return ping;
			case "heartbeat":
				return heartbeat;
			default:
				return null;
		}
	}
}

We chose the path "heartbeat" here. This concludes the server updates.

Heartbeat client

Now to our client implementation. We need a connector and a client-side service implementation. This time, we will allow our application to specify a listener to be called when a heartbeat is received, so we can do something with it:

import org.openzen.packetstreams.PacketStream;
import org.openzen.packetstreams.ServiceConnector;
import org.openzen.packetstreams.ServiceMeta;
import org.openzen.packetstreams.ServiceStream;
import org.openzen.packetstreams.io.BytesDataOutput;

public class HeartbeatServiceConnector implements ServiceConnector {
    private final int interval;
    private final HeartbeatServiceStream.Listener listener;

    public HeartbeatServiceConnector(int interval, HeartbeatServiceStream.Listener listener) {
        this.interval = interval;
        this.listener = listener;
    }

    @Override
    public byte[] connect(ServiceMeta meta) {
        BytesDataOutput output = new BytesDataOutput();
        output.writeVarUInt(interval);
        return output.toByteArray();
    }

    @Override
    public byte[] connectWithUpdatedMeta(ServiceMeta meta) {
        return connect(meta);
    }

    @Override
    public ServiceStream onConnected(PacketStream stream) {
        return new HeartbeatServiceStream(stream, listener);
    }
}

import org.openzen.packetstreams.BufferedServiceStream;
import org.openzen.packetstreams.PacketStream;
import org.openzen.packetstreams.io.BytesDataInput;

public class HeartbeatServiceStream extends BufferedServiceStream {
    private final Listener listener;

    public HeartbeatServiceStream(PacketStream stream, Listener listener) {
        super(stream);

        this.listener = listener;
    }

    @Override
    public void onConnected() {

    }

    @Override
    public void onReceived(byte[] packet) {
        BytesDataInput input = new BytesDataInput(packet);
        listener.onReceivedHeartbeat(input.readVarLong());
    }

    @Override
    public void onConnectionClosed(int reason, byte[] info) {

    }

    public interface Listener {
        void onReceivedHeartbeat(long serverTime);
    }
}

We can then make it connect when we open the client:

class MainClient {
	public static void main(String[] args) throws IOException {
		CryptoProvider crypto = JavaCrypto.INSTANCE;
		QPSPEndpoint client = new QPSPEndpoint(0, crypto);
		
		byte[] rootKey = Base64.decode("{SigningKey.txt}", Base64.DEFAULT);
		PinnedRootValidator rootValidator = new PinnedRootValidator(rootKey);
		ClientSession clientSession = client.connect("localhost", rootValidator);
		clientSession.whenEstablished(session -> {
			session.open("ping", new PingConnector());
			session.open("heartbeat", new HeartbeatServiceConnector(
				1000,
				time -> System.out.println("Heartbeat received")
			));
		});
		client.open();
	}
}

This example has a fixed interval of a second (1000 ms) and will print "Heartbeat received" whenever a message has been received.

This concludes the PacketStreams tutorials. Have fun!