BitTorrent has made distributing files trivial. That's awesome.
Live streaming, though, is still really really hard and takes a lot
of money. So, the idea is to adapt BitTorrent to streaming.
Since torrent files are chunked little bits anyway, this should be
"easy". Also, I don't want to have to be ready to start
streaming when it first comes on, necessarily, since one of the
things I like about downloading things is that I can't
"miss" them.
The basic idea is to have a swarm that has chunks and shares them
exactly the same way it ever does. Then, though, when the source
seeder has encoded a new chunk, they signal to the swarm that a new
chunk is ready. In general, this should look very similar to a swarm
where nobody has chunk 15, but then someone arrives who does. The
only difference is that chunk 15 didn't exist until now, and no one
was sure it was going to.
That's the basic idea. Then, when someone joins the swarm they can
choose to download every chunk that's been downloaded so far, thus
getting the stream from the beginning, while it's still being laid
down, or they can choose to just download the last chunk, and then
the new ones as they come in.
Then, when the stream is over, the source publishes an
"end" chunk that is a normal chunk, but signals that the
file is over now. The swarm, though, can continue to hang around and
seed, since we now have a swarm that's seeding the content already.
For example, it's an episode of a show. As the show is on, the
source is recording it and releasing chunks, and people who join at
any time while the show is airing can start downloading the chunks
that already exist. If the show starts on boring TV at 19:00, and
they show up at 19:23, they can still watch it from the beginning.
But, their client is also downloading the new chunks as they come
out, so by the time the show is over, they'll have all of it.
Now, the show is done airing, and it's just a regular torrent.
It's got a bunch of regular chunks, and someone who shows up an hour
after the show ends might not even know that it used to be dynamic.
So, the latency will be higher here, but I think that'd be fine. I
don't need to be really up-to-date, I just want to start it before
it's over. The way it currently is has two options:
- Wait for it to finish airing and then be uploaded
- Have a (crappy, sketchy) stream, but if you come in 4 minutes
late you missed the first 4 minutes like we've learned nothing since
antenna TV.
I like this better. Also, latency and stuff will very quite a bit
on chunk size. If the chunk size is the size of roughly 3 minutes
worth of video, then a chunk won't be published until at least 3
minutes have passed (since a given chunk is atomic)
Playing the file is up to the client. This doesn't have any kind
of smarts. It just allows one to start downloading a file before the
end of it is finalized. The same thing could be used for, like, the
output of `tar -cjf blah.tar.bz2 files`, to start allowing people to
download the tarball before the end is complete. It's up to the video
player to be able to play the start of a file without the end.
Certain formats (like mkv and mp4) are better at that than others
(like avi), but the protocol doesn't care.
This is a problem for more constant streams, like webcams or
something. It'd be nice to have those, but really I think I'd want
different tech for it. To be able to just jump in live, like I said
above, by downloading the most recent chunk and then every one after
that isn't likely to work with any format that I know of. The headers
and stuff need to come down no matter what. One could maybe make a
format that works that way, with a headers file and a "raw"
stream that you can do... I don't know.
So, I feel like this is less suited to webcams and more suited to
airing programs through torrents as they air. Anything that starts,
goes, then ends.
BitTorrent has something at http://live.bittorrent.com that
seems more like the solution to that last problem I was talking
about. They mention that it needs H.264 and stuff, which is way more
smarts than I want, but I think it's because they're going more for
the "jump into the stream" style stuff, which is harder to
do dumb. They'll probably also need a special player, etc. I don't
want that.
So, the one issue I can see, is "who gets to publish new
chunks". That seems like an obvious attack. So, I'm thinking
that as part of the streaming torrent metadata, a public-key is given
as well. Then, the "New chunk" message is signed with that
key. That way it can be forwarded through the network
peer-to-peerfully, but you know that the new chunk your neighbour
just told you about actually belongs to this stream.
It's been a while since I read the actual protocol spec, maybe
this is all super dumb and useless. I intend to look into that later,
maybe branch off of libtransmission, see if it is a workable idea.