Advanced bash pipes combine after split
Hello,
I have an encrypted archive, only part of the archive (first 10MB of the tar.gz) is actually encrypted, since the archive is quite large and asymmetric decription is slow. On the receiving side I want the download and decryption + inflating to be as quick as possible. That's why I'm using pipes, not download locally and then decrypt and then inflate. I intend to use tee: curl http://server/file | tee >(dd bs=1M count=10 | openssl... | ???) | dd bs=1M skip=10 | ??? -curl downloads the file -tee splits the stream: 1. first stream acts only on the first 10MB, decrypts using openssl 2. second stream acts only after the first 10MB ??? don't know how to go on from here Question: how do I join back these pipes in one pipe to input to "tar xzf -"? Thank you. |
Quote:
|
Even openssl symmetric encryption is slow (3.5MB/sec) compared to LZO (25MB/sec).
I'm looking now at named pipes, maybe that can solve the problem. |
I still haven't figured it out but, I've tried with named pipes:
mkfifo pipe dd if=pipe | lzop -d | tar xf - & #this should wait for input in the pipe and decompress now I have the two parts of the archive 1.lzo and 2.lzo, and I do: dd if=1.lzo of=pipe dd if=2.lzo of=pipe This should work but it does not! The pipe is closed after the first dd, and I get an unexpected EOF from lzop. How can I keep the pipe open? What good is a named pipe which closes after the first write to it? |
It can be done with named pipes
It can be done with named pipes:
mkfifo pipe1 mkfifo pipe2 cat pipe1 pipe2 | lzop -d | tar xf - & curl part1 |openssl > pipe1 curl part2 > pipe2 |
All times are GMT -5. The time now is 12:18 PM. |