Network Programming :: Lessons :: Streams
Output Streams
A major function of most network programs is moving bytes from one computer to another. This is essentially input and output. Java has a unique approach to input and output compared to languages like C and C++. I/O in Java is built on streams that use the same basic methods to read and write data. Streams are synchronous, which means when a programs asks a stream to read or write data it waits for that task to complete before proceeding with the rest of the program.
The basic output class in Java is java.io.OutputStream. The class and important method definitions are defined below.
public abstract class OutputStream public abstract void write(int b) throws IOException public void write(byte[] data) throws IOException public void write(byte[] data, int offset, int length) throws IOException public void flush() throws IOException public void close() throws IOException
Subclasses of OutputStream use these methods to write data to different destinations. A FileOutputStream uses these methods to write data into a file and a TelnetOutputStream uses these methods to write data onto a network connection. Some subclasses, such as the TelnetOutputStream, are hard to find as they are not in the Javadocs, but you will know how to use them since they are all based on OutputStream.
The most important method in OutputStream is write(int b), which takes an integer from 0 to 255 (an unsigned byte) as an argument and write the corresponding byte to the output stream. The code below will write 72-character lines of ASCII text. The printable ASCII characters have ASCII numbers between 33 and 126. The first line will start with character 33 (!) and end with character 104 (h). The next line will start with character 34 (") and end with character 105 (i). This pattern will continue indefinitely.
public static void streamCharacters(OutputStream out) throws IOException {
int firstCharacter = 33;
int numCharacters = 94;
int numCharactersPerLine = 72;
int start = firstCharacter;
while (true) /* Infinite Loop */
{
for (int i = start; i < start + numCharactersPerLine; i++)
{
out.write((i - firstCharacter) % numCharacters) + firstCharacter);
}
out.write('\r'); // Carriage return
out.write('\n'); // new line
start = ((start + 1) - firstCharacter) % numCharacters + firstCharacter;
}
}
Bytes in the code above are written out one at a time, but this is often inefficient. Since TCP packets contain at least 40 bytes of extra data that is a lot of extra information that has to pass through the network. It's a good idea to send multiple bytes at once if you have them ready to go using the write(byte[] data) or write(byte[] data, int offset, int length) methods such as in the modified code below.
public static void streamCharacters(OutputStream out) throws IOException {
int firstCharacter = 33;
int numCharacters = 94;
int numCharactersPerLine = 72;
int start = firstCharacter;
byte[] output = new byte[numCharactersPerLine + 2];
while (true) /* Infinite Loop */
{
for (int i = start; i < start + numCharactersPerLine; i++)
{
output[i - start] = (byte)((i - firstCharacter) % numCharacters) + firstCharacter);
}
output[72] = '\r'; // Carriage return
output[73] = '\n'; // new line
out.write(output);
start = ((start + 1) - firstCharacter) % numCharacters + firstCharacter;
}
}
The difference with the second streamCharacters method is that all of the bytes are stored in an array before they are written to the stream. If you are done writing data, it is important to flush the output stream to make sure all of the data is sent to the destination. Failing to flush your stream can lead to programs freezing and it is a low-cost operation so it is better to do it whether you need to or not. You can flush a stream using the flush() method.When you finish with a stream you can call the close() method to release any resources connected to that stream such as the network connection or local file. The example below works in Java 7, and will automatically call the close method for any objects declared in the try block:
try (OutputStream out = new FileObjectStream('data.txt')) {
// do stuff with the output stream...
} catch (IOException msg) {
System.err.println(msg.getMessage());
}
Input Streams
The basic input class in Java is java.io.InputStream. The class and important method definitions are defined below.
public abstract class InputStream public abstract int read() throws IOException public int read(byte[] input) throws IOException public int read(byte[] input, int offset, int length) throws IOException public long skip(long n) throws IOException public int available() throws IOException public void close() throws IOException
Subclasses of InputStream use the above methods to read data from certain sources. FileInputStream reads data from a file while TelnetInputStream reads data from a network connection. The most important method is the read() method with no arguments, which reads a single byte of data from the input stream's source and returns it as an integer between 0 and 255. A return value of -1 indicates the end of the stream. The read() method will wait and block execution of any subsequent code until a byte of data is available and ready to be read. The following code reads 10 bytes from the InputStream in a byte at a time:
byte[] input = new byte[10];
for (int i=0; i < input.length; i++)
{
int b = in.read();
if (b == -1)
break;
input[i] = (byte) b;
}
The last line that casts b to a byte casts it to a signed byte between -128 and 127. If you need to convert it to an unsigned byte, which is what the write method uses, you can do it with the following code:
int unsignedByte = signedByte >= 0 ? signedByte : 256 + signedByte;
Reading a byte at a time is just as inefficient as writing a byte at a time. The read(byte[] input) and read(byte[] input, int offset, int length) methods attempt to fill the input array from the input stream. This attempt may fail, though, especially over a network. All of the requested data may not have arrived so the read methods will return an integer that specifies how many bytes were actually read. The code below will continue to read until all the bytes have been read.
int bytesRead = 0;
int bytesToRead = 1024;
byte[] input = new byte[bytesToRead];
while (bytesRead < bytesToRead) {
int numBytes = in.read(input, bytesRead, bytesToRead - bytesRead);
if (numBytes == -1) // end of stream
break;
bytesRead += numBytes;
}
The available method is an alternative to the above approach as it tells you how many bytes are available to be read. At the end of a stream available will return 0. The skip method can skip data, but this is more useful for local files than network connections. Finally, you should call the close method when you are done with your input stream.
Filter Streams
Java provides filter classes that can be used to translate streams from raw bytes to other formats. This the example to the right, a compressed and encrypted text file arrives from the network where it is sent through the TelnetInputStream. The BufferedInputSream using buffering to speed up the process. The CipherInputStream decrypts the data while the GZIPInputStream decompresses the decrypted data. The text is finally read into the application using an InputStreamReader.
Every filter output stream has the same write(), close(), and flush() methods as OutputStream and every filter input stream has the same read(), close(), and available() methods as InputStream.
Filters are connected to streams through their constructors. The following code shows a BufferedInputStream initialized with a FileInputStream.
FileInputStream input = new FileInputStream("data.txt");
BufferedInputStream buf = new BufferedInputStream(input);
Streams can be chained together when you need to use methods in one input stream from another input stream. You can do so like the following example:
DataOutputStream output = new DataOutputStream(
new BufferedOutputStream(
new FileOutputStream("data.txt")
)
);
While you can use the methods from BufferedOutputStream and FileOutputStream in the example above you should only read/write from the last filter in the chain so you should only write to the DataOutputStream.
The BufferedOutputStream class stores written data in a protected byte array named buf until the buffer is full or the stream is flushed. Buffering network applications usually results in a great gain in performance. BufferedInputStream work similarly with a buf array that serves as the source for the stream's read() methods. Only when the buffer runs out of data does the stream read from the original source. Buffering input doesn't have the same performance gains as buffering output on a network, but rarely does it cause a problem. The buffered stream classes do not declare their own methods, but they do have two constructors each, which you can see below.
public BufferedInputStream(InputStream in) public BufferedInputStream(InputStream in, int bufferSize) public BufferedOutputStream(OutputStream out) public BufferedOutputStream(OutputStream out, int bufferSize)
The PrintStream class shouldn't be used in network applications because it is platform-dependent, but you have already encountered it since System.out is a PrintStream.
The DataInputStream and DataOutputStream classes contain methods for reading and writing Java's primitive types as well as strings in a binary format. The DataOutputStream class provides 11 methods:
public final void writeBoolean(boolean v) throws IOException public final void writeByte(int v) throws IOException public final void writeShort(int v) throws IOException public final void writeChar(int v) throws IOException public final void writeInt(int v) throws IOException public final void writeLong(long v) throws IOException public final void writeFloat(float v) throws IOException public final void writeDouble(double v) throws IOException public final void writeBytes(String s) throws IOException public final void writeChars(String s) throws IOException public final void writeUTF(String str) throws IOException
All data is written in big-endian format (check out this short lesson from the University of Maryland to find out the difference between big-endian and little-endian) and integers are written in two's complement form. The writeChars() method loops through the string and writes each character as a Unicode character (UTF-16). The writeBytes() method loops through the string but only writes the least significant byte, meaning information outside the Latin-1 character set may be lost. This method should typically be avoided unless you are working with a network protocol that specifies the ASCII encoding. The writeUTF() method includes the length of the string, which only makes it compatible with other Java programs.
The DataIntputStream class can read every data type that DataOutputStream can read. There are only 9 methods, though, since writeChars() and writeBytes() have no complement. Reading an array of bytes or chars can be handled by reading them one at a time.
public final boolean readBoolean() throws IOException public final byte readByte() throws IOException public final short readShort() throws IOException public final char readChar() throws IOException public final int readInt() throws IOException public final long readLong() throws IOException public final float readFloat() throws IOException public final double readDouble() throws IOException public final String readUTF() throws IOException
DataInputStream also has the normal multibyte read() methods and two readFully() methods that repeatedly read data into an array until the requested number of bytes has been read. This is helpful when you know exactly how many bytes you have to read.
public final int read(byte[] input) throws IOException public final int read(byte[] input, int offset, int length) throws IOException public final readFully(byte[] input) throws IOException public final int readFully(byte[] input, int offset, int length) throws IOException
Readers and Writers
HTTP and many modern protocols don't typically specify ASCII encoding and may used localized encodings such as KOI8-R Cyrillic, Big-5 Chinese, and ISO 8859-9. Java's native character set is Unicode's UTF-16 encoding. When you can't assume ASCII encoding bytes and characters can no longer be considered the same thing. The java.io.Reader abstract class and java.io.Writer abstract class specify the API by which characters are read and written.
The Writer class mirrors the java.io.OutputStream class, but it is not used directly since it is an abstract class. Write contains the following methods:
protected Writer() protected Writer(Object lock) public abstract void write(char[] text, int offset, int length) throws IOException public void write(int c) throws IOException public void write(char[] text) throws IOException public void write(String s) throws IOException public void write(String s, int offset, int length) throws IOException public abstract void flush() throws IOException public abstract void close() throws IOException
A subclass of Writer must at least override the three abstract methods. The actual bytes that are written depend on the encoding so the same input can result in different bytes depending on the encoding being used.
OutputStreamWriter is the most important subclass of Writer. It receives characters from a Java program and converts them into bytes depending on the specified encoding. It then write them onto an output stream. The constructor specifies the output stream to write to as well as the encoding to use.
public OutputStreamWriter(OutputStream out, String encoding) throws UnsupportedEncodingException
Valid encodings are listed on Oracle's Javadocs. If no encoding is specified, the default encoding for the platform is used. The default character encoding of Mac OS and most flavors of Linux is UTF-8, but the Windows character encoding varies depending on the region. In the United States Windows uses Windows-1252. Because of the different character encoding, it is preferable to always set your own encoding.
The Reader class mirrors the java.io.InputStream class and is abstract with two protected constructors. The Reader class should only be used through subclasses.
protected Reader() protected Reader(Object lock) public abstract int read(char[] text, int offset, int length) throws IOException public int read() throws IOException public int read(char[] text) throws IOException public long skip(long n) throws IOException public boolean ready() public boolean markSupported() public void mark(int readAheadLimit) throws IOException public void reset() throws IOException public abstract void close() throws IOException
InputStreamReader is the most important subclass of Reader. It can read bytes from an input stream and convert those bytes into characters according to a specified encoding. The constructor specifies the input stream and the encoding:
public InputStreamReader(InputStream in) public InputStreamReader(InputStream in, String encoding) throws UnsupportedEncodingException
After InputStreamReader and OutputStreamWriter are done, additional character filters can be layers using the java.io.FilterReader and java.io.FilterWriter classes. There are a number of subclasses for FilterReader and FilterWriter.
BufferedReader and BufferedWriter use an internal array of chars to perform the same function as BufferedInputStream and BufferedOutputStream. They have two constructors that allow you to set the underlying Reader or Writer as well as the buffer size. The default buffer size of 8,192 characters is used if you do not set the size.
public BufferedReader(Reader in, int bufferSize) public BufferedReader(Reader in) public BufferedWriter(Writer out) public BufferedWriter(Writer out, int bufferSize)
The BufferedReader class has a readLine() method that reads a single line of text and returns it as a string:
public String readLine() throws IOException
