io.TeeReader & io.MultiWriter
Linux tee command
The Linux tee command is another useful tool which lets you reading from standard input and then write that both to standard output and one or more files. The command is named after the T-splitter used in plumbing. It basically breaks the output of a program so that it can be both displayed and saved in a file. [source]
If I run:
ls | tee files.txt
I will get an output of all my directories printed to standard output and also save that output in the files.txt
. It's
a
pretty
handy way of reading some data immediately but also save that data in another file for later.
io.TeeReader
Go gives us this behaviour with the help of the io.TeeReader:
func TeeReader(r Reader, w Writer) Reader
TeeReader returns a Reader that writes to w what it reads from r. All reads from r performed through it are matched with corresponding writes to w. There is no internal buffering - the write must complete before the read completes. Any error encountered while writing is reported as a read error.
You might think that the TeeReader
is the same
as io.Copy and
indeed you can use the TeeReader to copy from a Reader to a Writer. But while io.Copy is just a copy operation from a
Reader to a Writer, io.TeeReader is a reader from which you can read again data from it.
io.TeeReader is useful when you want to copy some data but also want to have that copied data available for later use.
Let's clarify this with an example:
package main
import (
"bytes"
"fmt"
"io"
"os"
"strings"
)
func main() {
var r io.Reader = strings.NewReader("some io.Reader stream to be read\n")
teeReader := io.TeeReader(r, os.Stdout)
var buf bytes.Buffer
// when we copy from teeReader to buf, the data is also saved in os.Stdout
io.Copy(&buf, teeReader)
fmt.Println("data inside buffer:", buf.String())
}
Output:
some io.Reader stream to be read
data inside buffer: some io.Reader stream to be read
As you can see data from r io.Reader
was copied into buf
buffer but at the same time was saved in the os.Stdout
.
Don't forget, both os.Stdout
and bytes.Buffer
are readers here. Also notice
the .String() method we
called
for the first time on the bytes.Buffer
: it's quite a handy way of printing the strings inside a buffer:
func (b *Buffer) String() string
String returns the contents of the unread portion of the buffer as a string. If the Buffer is a nil pointer, it returns nil.
io.MultiWriter
When you know in advance all the places you want to save your data, you can achieve the same results with io.MultiWriter:
func MultiWriter(writers ...Writer) Writer
MultiWriter creates a writer that duplicates its writes to all the provided writers, similar to the Unix tee(1) command.
Let's try to write into two writers at once:
package main
import (
"bytes"
"fmt"
"io"
"strings"
)
func main() {
var (
buf1 bytes.Buffer
buf2 bytes.Buffer
)
// writing into mw will write to buf and ps.Stdou
mw := io.MultiWriter(&buf1, &buf2)
// r is the source of data(Reader)
r := strings.NewReader("some io.Reader stream to be read\n")
// write into mw from r
io.Copy(mw, r)
fmt.Println("data inside buffer1 :", buf1.String())
fmt.Println("data inside buffer2 :", buf2.String())
}
I like using io.MultiWriter
when I'm trying to find out what was written in a certain Writer. It's very helpful
for debugging when you have a function using a Writer and for some reason is too hard to get its contents. I just
connect another bytes.Buffer
to it and then check the contents of the buffer which will be the same as the contents
of my inaccessible Writer:
package main
import (
"bytes"
"fmt"
"io"
)
func main() {
buf := new(bytes.Buffer)
debug := io.MultiWriter(buf, weirdWriter)
go complicatedFunctionWithAWriter(weirdWriter)
// The contents of the buffer will be the same as in weirdWriter
fmt.Println(bug.String())
fmt.Println(bug.Bytes())
}
Conclusion
Now, when io.Copy is not enough you know about 2 other ways of copying data from a Reader into multiple writers. Pretty neat!