1
00:00:00,000 --> 00:00:03,010
Time we start with the next talk

2
00:00:03,560 --> 00:00:06,250
I welcome Richard Hartmann

3
00:00:06,250 --> 00:00:09,860
He is involved in Debian since many years

4
00:00:09,860 --> 00:00:14,310
and he became recently Debian Developer

5
00:00:14,310 --> 00:00:18,290
and he will talk about gitify your life.

6
00:00:18,290 --> 00:00:22,500
?, blogs, configs, data and backup. gitify everything

7
00:00:22,500 --> 00:00:24,310
Richard Hartmann

8
00:00:24,310 --> 00:00:25,370
Thank you. [applause]

9
00:00:30,980 --> 00:00:32,090
Thank you for coming

10
00:00:32,090 --> 00:00:35,240
expecially those who ? years attended all ?

11
00:00:37,240 --> 00:00:39,240
Short thing about myself

12
00:00:39,780 --> 00:00:42,460
As ? said I'm Richard Hartmann

13
00:00:42,460 --> 00:00:46,030
In my day job I am backbone manager at Globalways

14
00:00:46,030 --> 00:00:49,320
I'm involved in freenode and OFTC and...

15
00:00:49,320 --> 00:00:51,320
should I speak louder?

16
00:00:52,050 --> 00:00:53,320
I'm not...

17
00:00:55,990 --> 00:00:58,350
test, test... good back there?

18
00:01:01,130 --> 00:01:03,130
Can you turn up the volume a little bit?

19
00:01:05,570 --> 00:01:08,210
test, test... ok, perfect.

20
00:01:08,210 --> 00:01:13,080
Since about a week I've been a Debian Developer (yay)

21
00:01:13,080 --> 00:01:20,670
[applause] and I'm the author of vcsh.

22
00:01:20,670 --> 00:01:24,750
Raise of hands: who of you know what git is?

23
00:01:25,530 --> 00:01:26,750
perfect

24
00:01:26,750 --> 00:01:31,230
That's just as in ? perfect, we can skip it.

25
00:01:31,740 --> 00:01:34,060
Let's move to the first tool, etckeeper.

26
00:01:34,060 --> 00:01:37,250
Some of maybe most of this audience will have heard of it,

27
00:01:37,250 --> 00:01:45,960
it's a tool to basicly store your /etc in pretty much every version control system you can think of

28
00:01:46,370 --> 00:01:47,900
It's implemented in POSIX shell

29
00:01:47,900 --> 00:01:53,280
it autocommits every thing in /etc basically at every opportunity

30
00:01:53,280 --> 00:01:55,280
you may need to write excludes, for example

31
00:01:55,280 --> 00:01:58,050
before your network config ?

32
00:01:58,050 --> 00:02:00,050
but else, yeah, that's really cool

33
00:02:00,050 --> 00:02:01,100
the autocommit

34
00:02:01,100 --> 00:02:07,080
it hooks into most of the important or maybe even all of the important package management systems

35
00:02:07,080 --> 00:02:11,110
so when you install your packages, even on SuSE or whatever

36
00:02:11,110 --> 00:02:14,300
you can just have it commit automatically, which is very nice

37
00:02:14,900 --> 00:02:17,500
You can obviously commit manually

38
00:02:17,500 --> 00:02:19,950
if you for example change your X config

39
00:02:20,550 --> 00:02:23,050
it supports as I said various backends

40
00:02:23,050 --> 00:02:25,560
it's quite nice to recover from failures

41
00:02:25,560 --> 00:02:31,210
for example ? used it to recover from saturday's power outages

42
00:02:31,210 --> 00:02:36,300
because some servers lost stuff and with etckeeper you can just replay all the data which was...

43
00:02:36,300 --> 00:02:37,780
rather nice.

44
00:02:38,060 --> 00:02:39,260
Then there is bup.

45
00:02:39,260 --> 00:02:42,920
bup is a backup tool based on the git pack file format

46
00:02:42,920 --> 00:02:44,170
it's written in python

47
00:02:44,170 --> 00:02:45,650
it's very very fast

48
00:02:45,650 --> 00:02:47,320
and it's very space efficient.

49
00:02:47,740 --> 00:02:52,830
The author of bup managed to reduce his own personal backup size

50
00:02:52,970 --> 00:02:55,800
from 120 GiB to 45 GiB

51
00:02:55,800 --> 00:03:00,010
just by migrating away from rsnapshot over to bup

52
00:03:00,010 --> 00:03:02,050
which is quite good

53
00:03:02,050 --> 00:03:05,150
I mean, it's almost or a little bit more than a third, so

54
00:03:05,150 --> 00:03:06,400
very good

55
00:03:06,910 --> 00:03:09,640
This happens because it has built-in deduplication

56
00:03:09,640 --> 00:03:14,410
because obviously git pack files also have deduplication

57
00:03:14,690 --> 00:03:16,820
You can restore every single mount point

58
00:03:16,820 --> 00:03:19,000
or every single point in time

59
00:03:19,000 --> 00:03:23,400
every single backup can be monted as FUSE filesystem or a ? filesystem

60
00:03:23,400 --> 00:03:25,300
independently of each other

61
00:03:25,300 --> 00:03:28,220
so you can even compare different versions of what you have in your backups

62
00:03:28,590 --> 00:03:29,790
which again is very nice

63
00:03:29,790 --> 00:03:34,930
the one thing which is a real downside for serious deployments

64
00:03:36,230 --> 00:03:43,450
there is no way to get data out of your... archive or out of your backups

65
00:03:43,450 --> 00:03:46,690
which again is a direct consequence of using git pack files

66
00:03:46,690 --> 00:03:49,660
there is a branch which supports deleting old data

67
00:03:49,660 --> 00:03:52,990
but this is not in mainline and it hasn't been in mainline for...

68
00:03:54,240 --> 00:03:55,910
I think one or two years

69
00:03:55,910 --> 00:03:59,340
so I'm not sure if it will ever happen but...

70
00:03:59,340 --> 00:04:00,770
yeah

71
00:04:00,770 --> 00:04:02,770
at least in theory it would exist.

72
00:04:04,390 --> 00:04:09,850
Then for your websites, for your wikis, for your whatever there is ikiwiki.

73
00:04:10,820 --> 00:04:13,000
ikiwiki is a wiki compiler,

74
00:04:13,000 --> 00:04:14,300
as the name implies,

75
00:04:14,300 --> 00:04:19,440
and it converts various different files into HTML files

76
00:04:19,810 --> 00:04:20,870
it's written in Perl

77
00:04:20,870 --> 00:04:22,630
it supports various backends

78
00:04:22,630 --> 00:04:25,460
again most of the ones you can possibly think of

79
00:04:26,940 --> 00:04:28,750
oh, I can even slow down, good

80
00:04:30,410 --> 00:04:34,860
it's able to parse various markup languages, more on that on the next slide

81
00:04:35,320 --> 00:04:41,620
there are several different ways to actually edit any kind of content within ikiwiki

82
00:04:42,920 --> 00:04:46,350
it has templating support, it has CSS support

83
00:04:46,350 --> 00:04:51,900
these are quite extensive, but they may be improved, but that's for another time

84
00:04:51,900 --> 00:04:56,720
it acts as a wiki, as a CMS, as a blog, as a lot of different things

85
00:04:57,090 --> 00:05:02,510
it automatically generates RSS and Atom feeds for every single page, for every subdirectory

86
00:05:02,510 --> 00:05:05,840
so you can easily subscribe to topical content

87
00:05:05,840 --> 00:05:09,870
if you are for example only interested in one part of a particular page

88
00:05:09,870 --> 00:05:12,280
just subscribe to this part by RSS

89
00:05:12,280 --> 00:05:14,600
and you don't have to check if there updates for it

90
00:05:14,600 --> 00:05:19,740
which is very convenient to keep track of comments somewhere or something

91
00:05:20,060 --> 00:05:25,570
It supports OpenID, which means you dont have to go through all the trouble of...

92
00:05:25,940 --> 00:05:29,230
having a user database or doing very...

93
00:05:29,600 --> 00:05:31,450
or doing a lot of antispam measures

94
00:05:31,450 --> 00:05:34,650
because it turns out OpenID is relatively well...

95
00:05:34,650 --> 00:05:36,500
suited for just...

96
00:05:36,500 --> 00:05:39,470
stopping spam. For some reason, maybe they just

97
00:05:39,470 --> 00:05:40,810
haven't picked it up yet, I don't know

98
00:05:40,810 --> 00:05:43,870
but it's quite nice, because you don't have to do any actual work

99
00:05:44,240 --> 00:05:50,210
and people can still edit your content, and you can track back changes at least to some extent

100
00:05:51,780 --> 00:05:58,310
it supports various markup languages, the best ones, well, debatable, but in my opinion is Markdown

101
00:05:58,500 --> 00:06:06,510
it supports WikiText, reStructuredText, Textile and HTML and there are ikiwiki specific extensions

102
00:06:06,510 --> 00:06:12,020
for example normal wikilinks which are a lot more powerful than the normal linking style in MarkDown

103
00:06:12,020 --> 00:06:15,080
which kind of sucks, but... whatever

104
00:06:16,610 --> 00:06:22,580
it also supports directives, which basically tell ikiwiki to do special things with the page

105
00:06:22,580 --> 00:06:24,440
for example you can tag your blog pages

106
00:06:24,440 --> 00:06:26,940
or you can make...

107
00:06:27,210 --> 00:06:33,050
generate pages which automatically pull in content from different other pages and stuff like this.

108
00:06:33,050 --> 00:06:35,050
that's all done by directives.

109
00:06:38,280 --> 00:06:39,630
How does it work?

110
00:06:39,630 --> 00:06:44,530
You can edit webpages directly, if you want to, on the web

111
00:06:45,040 --> 00:06:49,770
then you will have a rebuild of the content

112
00:06:49,770 --> 00:06:52,320
but only the parts with changes

113
00:06:52,320 --> 00:06:55,790
so if you... hello?

114
00:06:55,790 --> 00:06:59,350
if you change only one single file it will only rebuild one single file

115
00:06:59,350 --> 00:07:03,850
if you change for example the navigation it will rebuild everything because obviously...

116
00:07:03,850 --> 00:07:05,850
it is used by everything.

117
00:07:15,700 --> 00:07:20,240
If it has to generate pages automatically, for example the index pages or something

118
00:07:20,240 --> 00:07:23,160
if you just create a new subdirectory, or if you have...

119
00:07:23,580 --> 00:07:25,850
if you have commands which have to appear on your site

120
00:07:25,850 --> 00:07:29,090
it will automatically generate those MarkDown files and commit them

121
00:07:29,090 --> 00:07:34,040
or you put them in your souce directory and you just commit them and...

122
00:07:34,270 --> 00:07:37,790
and have them part of your site, or you can autocommit them if you want.

123
00:07:37,790 --> 00:07:39,510
That's possible as well.

124
00:07:39,740 --> 00:07:45,950
You can obviously change... pull in changes in your local repository if you want to look at them

125
00:07:47,010 --> 00:07:49,190
Common uses would be public wiki...

126
00:07:49,190 --> 00:07:53,450
private notes, for just note keeping of your personal TODO list or whatever

127
00:07:54,140 --> 00:07:58,360
having an actual blog, which a lot of people in this room probably do

128
00:07:58,360 --> 00:08:04,280
that's, yeah, I mean a lot of people on Planet Debian have their blog on ikiwiki, for good reasons

129
00:08:04,890 --> 00:08:09,060
and an actual CMS for company websites or stuff

130
00:08:09,380 --> 00:08:12,070
which also tends to work quite well.

131
00:08:13,500 --> 00:08:21,700
The three main ways to interact with ikiwiki are webbased text editing, which is quite useful for new users, but is quite boring, in my opinion,

132
00:08:22,250 --> 00:08:28,090
there is also a WYSIWYG editor which is even more fancy for non-technical users

133
00:08:28,650 --> 00:08:32,720
and there is just plain old CLI-based editing way:

134
00:08:32,720 --> 00:08:38,600
just edit files and commit them back into repository pushes up and everything gets rebuilt automatically , which is...

135
00:08:38,830 --> 00:08:41,940
in my opinion the best way to interact with ikiwiki, because

136
00:08:41,940 --> 00:08:45,600
you are able to stay on the command line and simply push out your...

137
00:08:45,600 --> 00:08:50,460
your stuff onto the web and you don'tactually have to leave the command line

138
00:08:50,780 --> 00:08:53,100
which is pretty kinda neat.

139
00:08:54,070 --> 00:08:57,080
There are also some more advanced use cases

140
00:08:59,260 --> 00:09:02,640
as I said you can interface with the source files directly

141
00:09:02,640 --> 00:09:04,070
you can maintain...

142
00:09:04,540 --> 00:09:06,070
something is wrong

143
00:09:06,070 --> 00:09:10,420
for example you can maintain your your wiki and your docs and your...

144
00:09:10,420 --> 00:09:12,420
source code in one single directory

145
00:09:12,830 --> 00:09:15,280
and it would simply...

146
00:09:15,280 --> 00:09:18,800
and simply have parts of your subdirectory structure rendered.

147
00:09:18,800 --> 00:09:20,800
for example git-annex does this

148
00:09:20,800 --> 00:09:24,310
there is a doc directory, which is rendered to the website

149
00:09:24,590 --> 00:09:26,720
but is also part of the normal source directory

150
00:09:26,720 --> 00:09:30,750
which means that everybody who checks out a copy of the repository

151
00:09:30,750 --> 00:09:34,320
will have the complete forum, bug reports, TODO lists

152
00:09:34,320 --> 00:09:35,150
user comments,

153
00:09:35,240 --> 00:09:40,150
everything on their local filesystem, without having to leave - again - their command line,

154
00:09:40,150 --> 00:09:48,720
which doesn't break media, and so is just very convenient to have one single resource for everything regarding one single program.

155
00:09:49,690 --> 00:09:52,800
And another nice thing is if you create different branches

156
00:09:52,800 --> 00:09:58,910
for preview, staging areas you can have workflows where some people are just allowed to create ...

157
00:09:58,910 --> 00:10:05,070
pages, other people then look over those pages and merge them back into master and then push them on the website

158
00:10:05,390 --> 00:10:08,030
which basically allows you to...

159
00:10:08,590 --> 00:10:13,640
to have content control or real publishing workflow, if you have a need to do this

160
00:10:15,630 --> 00:10:17,900
Next stop: git-annex.

161
00:10:18,590 --> 00:10:20,490
The beef.

162
00:10:22,060 --> 00:10:28,690
It's basically a tool to manage files with git without checking those files into git

163
00:10:29,800 --> 00:10:31,980
?

164
00:10:34,660 --> 00:10:35,680
Yeah, what is git-annex?

165
00:10:35,680 --> 00:10:36,380
It's based on git,

166
00:10:36,380 --> 00:10:39,480
it maintains the metadata about files,

167
00:10:39,480 --> 00:10:43,410
as in location, and file names and everything, in your git repository

168
00:10:43,690 --> 00:10:48,930
but it doesn't actually maintain the file content within the git repository

169
00:10:48,930 --> 00:10:49,900
more on that later

170
00:10:49,900 --> 00:10:53,050
this saves a lot of time and space.

171
00:10:54,300 --> 00:10:59,160
You still able to use any git-annex repository as a normal git repository

172
00:10:59,160 --> 00:11:02,310
which ? means you're even able to have a mix of...

173
00:11:02,310 --> 00:11:05,090
for example, say, all your ? files

174
00:11:05,090 --> 00:11:07,960
should be maintained by normal git,

175
00:11:07,960 --> 00:11:12,170
and then you have all the merging which git does for you and everything

176
00:11:12,170 --> 00:11:13,930
and then you have for example your photographs,

177
00:11:13,930 --> 00:11:15,930
or your videos for web publishing

178
00:11:15,930 --> 00:11:18,840
which are maintained in the annex

179
00:11:18,840 --> 00:11:23,660
which means you don't have to have a copy of those files in each and every single location

180
00:11:26,020 --> 00:11:31,490
A very nice thing about git-annex is that it's written with very low bandwidth and flaky connections in mind

181
00:11:32,320 --> 00:11:35,980
quite a lot of you will know that Joey lives basically in the middle of nowhere

182
00:11:35,980 --> 00:11:40,470
which is a great thing to be forced to write really efficient code

183
00:11:40,930 --> 00:11:43,440
which doesn't use a lot of data, and that shows:

184
00:11:43,810 --> 00:11:44,320
it's really quick

185
00:11:44,320 --> 00:11:47,600
and even if you had a really really bad connection

186
00:11:47,600 --> 00:11:49,600
in backwaters or whatever...

187
00:11:49,600 --> 00:11:52,420
during holidays or during normal living

188
00:11:52,790 --> 00:11:56,130
it's still able to transfer the data which you need to transfer,

189
00:11:56,130 --> 00:11:58,130
it's very very nice

190
00:11:58,130 --> 00:12:02,100
There are various workflows: we'll see four of them in a few minutes

191
00:12:03,630 --> 00:12:08,630
So. It's written in Haskell, so it's probably strongly typed and nobody can write patches for it

192
00:12:11,130 --> 00:12:14,280
it uses rsync to actually transfer the data,

193
00:12:14,280 --> 00:12:17,010
which means it doesn't try to reinvent any wheels

194
00:12:17,010 --> 00:12:24,240
it's really just based on top of established and well know and well debugged programs

195
00:12:24,240 --> 00:12:28,820
In indirect mode, which in my personal opinion is the better mode,

196
00:12:28,820 --> 00:12:30,300
what it does is

197
00:12:30,300 --> 00:12:36,190
it moves the actual files into a different location, namely .git/annex/objects

198
00:12:36,650 --> 00:12:41,930
it then makes those files read only, so you cannot event accidentally delete those files

199
00:12:41,930 --> 00:12:46,560
even if you rm -f them, it will still tell you no, I can't delete them,

200
00:12:46,560 --> 00:12:48,090
which is very secure

201
00:12:49,010 --> 00:12:52,210
may be incovenient, but you can work on this

202
00:12:52,210 --> 00:12:56,790
it replaces those files with symlinks of the same name, and those just point at the object

203
00:12:56,790 --> 00:12:59,850
and if there is an object behind this symlink or not...

204
00:12:59,850 --> 00:13:06,330
that basically returns whether you are able on this particular machine, or in this particular repository

205
00:13:06,800 --> 00:13:13,420
but you will definitely have the informations about the name of the file, the theorethical location of the file...

206
00:13:13,420 --> 00:13:16,570
the hash of the file will be in every single repository

207
00:13:17,080 --> 00:13:18,570
There is also a direct mode

208
00:13:19,300 --> 00:13:22,310
initially mainly written for windows and Mac OS X

209
00:13:22,310 --> 00:13:24,770
because Windows just doesn't support symlinks properly

210
00:13:24,770 --> 00:13:27,820
and OS X was supporting symlinks,

211
00:13:27,820 --> 00:13:32,040
apparently has lots of developers who think it is a great idea to follow symlinks...

212
00:13:32,040 --> 00:13:35,190
and display the actual target of the symlink instead of the symlink

213
00:13:35,190 --> 00:13:39,220
so you have cryptic filenames which are very hard to deal with

214
00:13:39,490 --> 00:13:46,160
obviously people who are used to GUI tools which then only display really really cryptic names ?

215
00:13:46,160 --> 00:13:49,590
so there is direct mode which doesn't do the symlink stuff

216
00:13:49,590 --> 00:13:53,390
it basically rewrites the files on the fly

217
00:13:53,390 --> 00:13:57,650
git still thinks it would be managing symlinks, but...

218
00:13:57,650 --> 00:14:03,860
git-annex just pulls them up from under git, and pushes in the actual content.

219
00:14:04,690 --> 00:14:08,760
You keep on nodding, so... I'm probably doing good

220
00:14:10,480 --> 00:14:14,320
and if you want you can always delete old data, or you can keep it...

221
00:14:14,320 --> 00:14:16,730
or you can just... for example what I'm doing:

222
00:14:16,730 --> 00:14:20,060
you can have one or two machines which slurp up all your data...

223
00:14:20,060 --> 00:14:26,410
and have an everlasting archive of everything which you've ever put into your annexes...

224
00:14:26,410 --> 00:14:30,210
and other machines, for example laptops with smaller SSDs

225
00:14:30,580 --> 00:14:34,100
those just have the data which you are actually interested in at the moment

226
00:14:35,760 --> 00:14:38,170
How does this work in the background?

227
00:14:38,310 --> 00:14:40,770
Each repository has a UUID

228
00:14:41,280 --> 00:14:45,860
It also has a name, which makes it easier for you to actually interact with the repository...

229
00:14:45,860 --> 00:14:48,960
but in the background it's just the UUID for obvious reasons...

230
00:14:48,960 --> 00:14:55,350
because it just makes ? and synchronization easy, period

231
00:14:55,350 --> 00:14:58,970
It's also tracking informations in a special branch called git-annex

232
00:14:59,430 --> 00:15:02,760
this branch means that all...

233
00:15:05,640 --> 00:15:11,100
this branch ? every single repository has full and complete informations...

234
00:15:11,100 --> 00:15:16,330
about all files, about the locations of all files, about the last status of those files...

235
00:15:16,330 --> 00:15:18,740
if those files have been added to some repository

236
00:15:18,740 --> 00:15:19,390
or they have been deleted,

237
00:15:19,390 --> 00:15:22,260
or if they are being over there forever

238
00:15:22,260 --> 00:15:30,640
so in every single repository you can just lookup the status of this file or of all files in all of your repositories

239
00:15:30,640 --> 00:15:33,420
which is, yeah, convenient

240
00:15:33,930 --> 00:15:37,780
The tracking information is very simple

241
00:15:37,780 --> 00:15:40,930
and it's designed to be merged very...

242
00:15:40,930 --> 00:15:43,430
it's a little bit more complicated than applying union merge,

243
00:15:43,430 --> 00:15:46,210
but basically what it does is it adds a timestamp

244
00:15:46,720 --> 00:15:52,600
and tells if the file is there or not and it has the UUID of the repository

245
00:15:53,060 --> 00:15:57,040
and from this informations, along with the timestamps you can simply reproduce...

246
00:15:57,140 --> 00:16:03,620
the whole lifecycle of your files through your whole cloud of git-annex repositories

247
00:16:03,620 --> 00:16:05,620
in this one particular annex.

248
00:16:06,540 --> 00:16:09,640
One really nice which you can do is...

249
00:16:09,640 --> 00:16:12,880
if you are on the command line, which again in my opinion is the better mode...

250
00:16:12,880 --> 00:16:14,880
you can simply run git-annex sync

251
00:16:14,880 --> 00:16:16,680
which basically does a commit...

252
00:16:16,960 --> 00:16:20,290
oh, it does a git-annex add, then it does a commit,

253
00:16:20,290 --> 00:16:24,090
then it merges from the other repositories

254
00:16:24,090 --> 00:16:27,380
into your own master, into your own git-annex branch

255
00:16:27,380 --> 00:16:29,380
then it merges the log files

256
00:16:29,380 --> 00:16:31,380
that's where the git-annex branch comes in

257
00:16:31,380 --> 00:16:34,090
and then it pushes to all other known repositories

258
00:16:34,090 --> 00:16:42,200
which is basically a one-shot command to syncronize all the metadata about all the files with all the other repositories

259
00:16:42,520 --> 00:16:44,610
and it takes no time at all

260
00:16:44,610 --> 00:16:46,610
given a network connection

261
00:16:47,800 --> 00:16:52,020
Data integrity is something which is very important for...

262
00:16:52,390 --> 00:16:57,200
yeah, for all of the tools, but git-annex was really designed with data integrity in mind

263
00:16:57,570 --> 00:17:03,920
by default it uses a SHA-2 256 with file extension...

264
00:17:03,920 --> 00:17:08,130
to store the objects, so it renames the file to its own shasum

265
00:17:08,130 --> 00:17:12,670
which allows you to always verify the data even without git-annex

266
00:17:12,900 --> 00:17:16,609
you are able to say by means of globbing...

267
00:17:16,609 --> 00:17:22,440
which files, or which directory, or which types of files should have how many copies in different repositories

268
00:17:22,440 --> 00:17:24,200
so for example what I do:

269
00:17:24,200 --> 00:17:27,859
all my raw files, all theraw photographs are in at least three different locations,

270
00:17:28,000 --> 00:17:31,800
all the JPEGs are only in two, because JPEGs can be regenerated

271
00:17:31,800 --> 00:17:32,910
raws can not.

272
00:17:34,480 --> 00:17:37,540
All remotes and all special remotes can always be verified

273
00:17:37,540 --> 00:17:40,970
with special remotes this may take quite some bandwidth

274
00:17:40,970 --> 00:17:46,250
with actual normal git-annex remotes you run the verification locally

275
00:17:46,250 --> 00:17:52,360
and just report back the results with obviously saves a lot of bandwidth and transfer time

276
00:17:53,940 --> 00:17:58,100
verification obviously takes the amount of requires copies into account

277
00:17:58,100 --> 00:18:00,510
so if you would have to have 3 different copies

278
00:18:00,510 --> 00:18:04,680
and your whole repository cloud only has 2, it will complain

279
00:18:04,680 --> 00:18:09,130
it will tell you "yes, checksum is great, but you don't have enough copies, please do something about it".

280
00:18:10,650 --> 00:18:14,960
and even if you ? right now, delete all copies from git annex

281
00:18:14,960 --> 00:18:18,760
you would still be able to get all your data out of git annex

282
00:18:18,760 --> 00:18:23,580
because what it boils down to, in indirect mode, it's just symlinks to other objects

283
00:18:23,580 --> 00:18:27,740
these objects have their own checksum as their file name

284
00:18:27,740 --> 00:18:30,710
so you'll even be able to verify, without git-annex,

285
00:18:30,710 --> 00:18:33,020
just by means of a little bit of shell scripting,

286
00:18:33,020 --> 00:18:35,290
that all your files are correct,

287
00:18:35,480 --> 00:18:38,950
that you don't have any bit flips or anything on your local disk.

288
00:18:40,020 --> 00:18:44,280
direct mode doesn't really need a recovery ?, because...

289
00:18:44,790 --> 00:18:48,310
the actual file is just in place of the symlink

290
00:18:52,340 --> 00:18:54,980
but on the other hand you won't be...

291
00:18:55,350 --> 00:18:59,420
you still need to look at the git-annex branch to determine the actual checksums

292
00:18:59,420 --> 00:19:02,010
which you wouldn't have to do with the indirect mode.

293
00:19:03,220 --> 00:19:07,850
There are a lot of special remotes. And what are special remotes?

294
00:19:07,850 --> 00:19:11,420
these are able to store data in non git-annex remotes

295
00:19:11,420 --> 00:19:16,050
because, let's face it, on most servers, or most servers where you could store data

296
00:19:16,050 --> 00:19:19,340
you aren't actually able to get a shell and execute commands

297
00:19:19,340 --> 00:19:22,070
you can just push data to it, you can receive data

298
00:19:22,070 --> 00:19:25,360
but you cannot actually execute anything on this computer.

299
00:19:26,790 --> 00:19:28,970
That's what special remotes are for.

300
00:19:30,400 --> 00:19:33,690
All special remotes support encrypted data storage

301
00:19:33,690 --> 00:19:37,120
so you just gpg encrypt your data and then send it off

302
00:19:37,120 --> 00:19:41,660
which means that the remotes can only see the file names

303
00:19:41,660 --> 00:19:46,200
but they cannot see anything else about the contents of your files

304
00:19:46,200 --> 00:19:51,800
obviously you don't want to trust amazon or anyone to store your plain text data

305
00:19:51,800 --> 00:19:53,800
that would just be stupid

306
00:19:53,800 --> 00:19:59,260
There is a hook system, which allows you to write a lot of new special remotes

307
00:19:59,260 --> 00:20:05,880
you'll see a list of... quite an extensive list of stuff in a second

308
00:20:05,880 --> 00:20:10,650
Normal, built-in, special remotes which are supported by haskell out of the box

309
00:20:11,020 --> 00:20:12,550
by git-annex out of the box

310
00:20:12,550 --> 00:20:15,470
and actually implemented in haskell

311
00:20:15,700 --> 00:20:21,860
are Amazon Glacier, Amazion S3, bup, directory — a normal directory on your system

312
00:20:21,860 --> 00:20:26,720
rsync, webdav, http or ftp and the hook system

313
00:20:28,390 --> 00:20:31,680
there is a guy who brought most of those

314
00:20:32,090 --> 00:20:37,000
we can support archive.org, IMAP, box.com, Google Drive... you can read them yourself, I mean...

315
00:20:37,140 --> 00:20:41,260
but those are quite a lot of different special remotes, if you...

316
00:20:41,260 --> 00:20:49,410
already have storage on any of those services, just start pushing encrypted data to it if you want, and you're basically done.

317
00:20:51,680 --> 00:20:54,830
There is an ongoing project called the git-annex assistant

318
00:20:55,110 --> 00:20:58,630
last year, and I think this year it just ended, didn't it?

319
00:20:59,650 --> 00:21:05,300
so, pretty much exactly one year ago Joey has started to to raise funds

320
00:21:05,390 --> 00:21:12,060
by means of a kickstarter to just focus on writing git-annex assistant for a few months

321
00:21:12,660 --> 00:21:15,300
he got so much that he could do it for a whole year

322
00:21:15,300 --> 00:21:22,620
and he's just restarted the whole thing with his own fundraising campaign without kickstarter and he got another full year

323
00:21:24,060 --> 00:21:32,530
yeah... are you still accepting funds?

324
00:21:33,600 --> 00:21:38,000
ok, so, if you use it at least consider donating

325
00:21:38,000 --> 00:21:44,110
because honesty you can't write patches for it anyway, because it's in haskell, so...

326
00:21:44,110 --> 00:21:48,600
that's... the other means of actually contributing

327
00:21:53,090 --> 00:21:57,490
git-annex boils down to be a daemon, which runs in the background

328
00:21:58,100 --> 00:22:03,190
and keeps track of all of your files, of newly added files

329
00:22:03,190 --> 00:22:09,490
and then starts transferring those files, if configured to do so

330
00:22:09,490 --> 00:22:14,580
it starts transferring files to other people or to other repositories

331
00:22:15,050 --> 00:22:17,820
this is all managed by means of a web gui

332
00:22:17,820 --> 00:22:25,880
which in turns means that it's really, well, not easy, but easier to port to for example windows or android

333
00:22:25,880 --> 00:22:28,240
which both work, to some extent

334
00:22:28,890 --> 00:22:33,010
not fully, but they are useful, or useable, more or less

335
00:22:34,270 --> 00:22:39,920
at least on android it's really quite good, I couldn't test it on windows, because...

336
00:22:41,120 --> 00:22:44,870
and it also makes it accessible for non technical users

337
00:22:44,870 --> 00:22:50,010
so for example if you want to share some of your photographs with your parents

338
00:22:50,010 --> 00:22:54,320
or with friends, or if you want to share, I don't know, videos with other people

339
00:22:54,320 --> 00:22:56,540
you just put them into one of those repositories

340
00:22:56,540 --> 00:23:02,280
and even those non-technical people just magically see stuff appear in their own repository

341
00:23:02,280 --> 00:23:04,280
and can just pull the data if they want to

342
00:23:04,280 --> 00:23:08,170
or if you configured it to do so, it would it would even transfer all the data automatically

343
00:23:08,910 --> 00:23:13,490
which is... it's ?

344
00:23:14,700 --> 00:23:20,020
It supports content notifications, but not content transfer

345
00:23:20,020 --> 00:23:22,240
by means of xmpp or jabber

346
00:23:22,240 --> 00:23:26,510
which used to work quite well with google talk, I think it's not...

347
00:23:27,520 --> 00:23:29,150
oh, it still works, ok

348
00:23:29,560 --> 00:23:36,740
at least at the moment, we'll see when they just ? google ? with google+, but...

349
00:23:37,810 --> 00:23:43,220
at least at the moment it still works, if you have a google account you can simply transfer all your data

350
00:23:43,220 --> 00:23:49,250
you can transfer the metadata about your data, you cannot actually transfer the files through jabber

351
00:23:49,250 --> 00:23:54,060
but that's probably something which will happen within the next year

352
00:23:54,850 --> 00:23:58,040
there are quite ? rulesets for content distribution

353
00:23:58,040 --> 00:24:03,970
so for example I can show you...

354
00:24:03,970 --> 00:24:10,550
you can say "put all raw files into this archive, and all jpegs on my laptop", or whatever

355
00:24:10,550 --> 00:24:15,740
or "if I still have more than 500 GB free on this please put data in

356
00:24:15,740 --> 00:24:21,480
and as soon as only have 20 left stop putting data into this one repository"

357
00:24:21,480 --> 00:24:24,070
which obviously is quite convenient

358
00:24:24,070 --> 00:24:27,640
as I said there is a windows port, and now on to usecases.

359
00:24:28,150 --> 00:24:30,230
First usecase: the archivist.

360
00:24:30,690 --> 00:24:34,030
What the archivist does is: basically he just collects data

361
00:24:34,030 --> 00:24:37,600
either to ? or just to collect

362
00:24:38,100 --> 00:24:43,250
and if you have this usecase what you probably want to do, you want to have offline disks

363
00:24:43,340 --> 00:24:47,370
to store at your mom's, or to put into a drawer

364
00:24:47,370 --> 00:24:52,790
or just you don't have enough sata ports in your computer because you just have so much data

365
00:24:53,200 --> 00:24:59,550
so, what you can do is you can just push this data to either connected machines or to disconnected drives...

366
00:24:59,550 --> 00:25:02,140
or to some webservice, and just store data

367
00:25:02,140 --> 00:25:06,400
but normally you would have the problem of keeping track of where your data lives

368
00:25:06,400 --> 00:25:08,760
if it's still ok, if it's still there, everything.

369
00:25:08,760 --> 00:25:16,270
With git-annex you can automate all this administrative side of archiving your stuff.

370
00:25:17,240 --> 00:25:22,330
Even if you only have one of those disks, if they're a proper remote...

371
00:25:22,330 --> 00:25:27,060
you'll have full informations about all the data in your annex cloud up to this point

372
00:25:27,060 --> 00:25:32,800
so even if you only pull out one random disk you still have informations on all the other disks on this one disk

373
00:25:32,800 --> 00:25:36,130
which obviously is a nice thing.

374
00:25:36,600 --> 00:25:38,130
Media consumption.

375
00:25:38,130 --> 00:25:44,890
Let's say you pull a video of this talk, or you get some slides...

376
00:25:44,890 --> 00:25:47,810
maybe also from this talk, you can get some podcasts...

377
00:25:47,810 --> 00:25:52,950
and git-annex has become a native podcatcher quite recently, I thing two or three weeks ago

378
00:25:52,950 --> 00:25:55,860
which means you don't even have a separate podcatcher

379
00:25:56,650 --> 00:26:02,300
you just tell git-annex "this is all of my rss feeds" and it will just pull in all the content,

380
00:26:02,860 --> 00:26:08,320
Then you can synchronize all this data for example to your cellphone, or your tablet, or whatever

381
00:26:08,320 --> 00:26:14,250
consume the data on any of your devices, even if you have 10 copies of this particular podcast

382
00:26:14,250 --> 00:26:16,940
because you didn't get around to listen to it on your computer...

383
00:26:17,170 --> 00:26:20,220
and you didn't get around to listen to it on your cellphone

384
00:26:20,220 --> 00:26:21,890
but then on your tablet you did listen to it

385
00:26:21,890 --> 00:26:24,760
you have three copies of this file which you don't need anymore...

386
00:26:24,760 --> 00:26:27,870
because you have listened to the content and you don't care about the content anymore

387
00:26:27,870 --> 00:26:33,610
what you do is you drop this content on one random repository

388
00:26:33,610 --> 00:26:37,730
and this information that you have dropped the actual content,

389
00:26:37,730 --> 00:26:41,710
not the metadata about the content, but the actual content, you don't need the content anymore...

390
00:26:41,710 --> 00:26:47,270
will slowly propagate to all of the annexes and if they have the data they will also drop the data

391
00:26:47,270 --> 00:26:52,830
so you don't have to really care about keeping track of those things

392
00:26:52,830 --> 00:26:56,490
you can simply have this message propagate

393
00:26:56,900 --> 00:27:00,520
do you want to comment? can someone give Joey a microphone?

394
00:27:07,180 --> 00:27:09,500
Just a minor correction

395
00:27:09,500 --> 00:27:11,630
it doesn't propagate that you've dropped the content

396
00:27:11,630 --> 00:27:15,470
but you can move it around in ways that have exactly the effect you described

397
00:27:15,570 --> 00:27:22,380
? get the wrong idea that if you accidentally remove one thing it will vanish from everything ?

398
00:27:22,610 --> 00:27:25,520
but if you deliberately drop the content and tell the annex...

399
00:27:25,520 --> 00:27:27,520
no. that's not how it works.

400
00:27:27,840 --> 00:27:29,830
I want to talk about it later, but it's...

401
00:27:29,830 --> 00:27:31,640
you looked at the slides, but...

402
00:27:31,640 --> 00:27:33,300
sorry, ?

403
00:27:34,650 --> 00:27:37,010
He watches for everything which is ?

404
00:27:46,830 --> 00:27:55,210
Next thing, if you are on the road, and one usecase which is probably quite common: taking pictures while you are on the road ?

405
00:27:55,490 --> 00:27:57,670
You take your pictures, you save them to your annex

406
00:27:57,670 --> 00:28:01,000
where you are able to store them back to your server or wherever

407
00:28:01,000 --> 00:28:06,930
if you want to, and even if for example one disk gets ?

408
00:28:06,930 --> 00:28:09,060
and you lose part of your content,

409
00:28:09,060 --> 00:28:13,970
you'll still at least be able to have an overview of what content used to be in your annex

410
00:28:13,970 --> 00:28:20,540
and if you then pull out your old SD card and see "oh, that photo is still there" you can simply reimport it and it will magically reappear.

411
00:28:20,540 --> 00:28:21,930
What it also does is:

412
00:28:21,930 --> 00:28:23,880
if you have a very tiny computer with you

413
00:28:23,880 --> 00:28:29,160
you can, as soon as you are at an internet cafe, just sync up with your server or your storage, whatever

414
00:28:29,160 --> 00:28:34,110
and push out the data to your remotes

415
00:28:34,110 --> 00:28:38,510
which then means you'll have two or three or five copies of the data

416
00:28:38,510 --> 00:28:41,480
and git-annex keeps track of what is where for you

417
00:28:41,480 --> 00:28:45,410
so you don't have to worry about copying stuff around.

418
00:28:47,590 --> 00:28:50,830
And then there is one personal usecase, for photographs

419
00:28:51,670 --> 00:28:55,790
I have a very specific way of organizing my photographs

420
00:28:55,790 --> 00:28:58,150
my wife disagrees violently

421
00:28:59,720 --> 00:29:03,380
she likes to do her photo storage in a completely different way

422
00:29:03,380 --> 00:29:04,730
she doesn't care about the raw files

423
00:29:04,730 --> 00:29:12,460
and she doesn't care about all the documentation pictures of signposts or whatever which I just took to remember which cities we went through

424
00:29:12,460 --> 00:29:18,900
so what she can do is she can simply delete the actual files or ? the symlink of this file

425
00:29:18,900 --> 00:29:21,720
and it will disappear from her own annex

426
00:29:21,720 --> 00:29:24,270
she can then commit all this

427
00:29:24,270 --> 00:29:29,780
normally if she would sync back the data I would also have the same layout, which I don't want

428
00:29:29,780 --> 00:29:33,670
expecially since she tends to rename everything a lot

429
00:29:33,670 --> 00:29:38,900
but what I did, I set up a rebasing branch on top of my normal git-annex repository

430
00:29:38,900 --> 00:29:43,070
so what she gets is: she has her own view of the whole data

431
00:29:43,070 --> 00:29:44,600
or the part she cares about

432
00:29:44,830 --> 00:29:47,060
and when I add new content

433
00:29:47,060 --> 00:29:51,040
she will see the new content, she will rearrange the content however she pleases

434
00:29:51,040 --> 00:29:52,710
but as it's a rebasing branch

435
00:29:52,710 --> 00:29:56,360
all her changes will always be replayed on top of master

436
00:29:58,730 --> 00:30:02,010
so she has her own view, and I don't even notice her own view

437
00:30:02,010 --> 00:30:08,270
but even if she uses one of the other computers she would have the same view which she herself has

438
00:30:08,270 --> 00:30:11,600
so basically she has her own view all of the data

439
00:30:11,600 --> 00:30:14,570
This is very convenient to keep the peace at home.

440
00:30:16,880 --> 00:30:18,830
Next topic: vcsh.

441
00:30:20,400 --> 00:30:23,270
Most of you here probably have some sort of system...

442
00:30:23,270 --> 00:30:26,610
where they have one subversion or cvs or whatever repository

443
00:30:26,610 --> 00:30:29,620
and they have it somewhere in their home directory

444
00:30:29,620 --> 00:30:35,960
you symlink into various places in your home directory, and it kind of keeps working so you don't throw it away, but...

445
00:30:35,960 --> 00:30:39,250
to be honest it sucks. Here is why.

446
00:30:40,550 --> 00:30:43,090
Or, here's why in a second.

447
00:30:43,510 --> 00:30:46,980
vcsh is implemented in POSIX, which is very very portable

448
00:30:46,980 --> 00:30:52,400
it's based on git, but it's not directly git

449
00:30:52,400 --> 00:30:57,450
The one thing which git is not able to do is maintain several different working copies into one dicrrectory

450
00:30:57,450 --> 00:31:00,320
which is a safety feature, more on that later

451
00:31:00,320 --> 00:31:05,790
but this really sucks if you want to maintain your mplayer, your shell, your whatever configuration

452
00:31:05,790 --> 00:31:10,790
in your home directory, which is the obvious and only real place where it makes sense to put your configuration

453
00:31:10,790 --> 00:31:14,080
you don't want to put it into dot-dot-files and then symlink back

454
00:31:14,080 --> 00:31:18,010
you want to have it in your home directory as actual files.

455
00:31:18,010 --> 00:31:21,070
So, vcsh uses fake bare git repositories

456
00:31:21,070 --> 00:31:22,830
again, more on that on the next slide

457
00:31:22,830 --> 00:31:25,240
and it's basically a wrapper around git

458
00:31:25,240 --> 00:31:30,930
which makes git do stuff which it normally wouldn't do

459
00:31:30,930 --> 00:31:36,120
and it has a quite extensible and useful hook system which ? will care about

460
00:31:36,910 --> 00:31:42,370
Whith a normal git repository you have two really defining variables within git

461
00:31:42,370 --> 00:31:44,370
you have the work tree

462
00:31:44,370 --> 00:31:46,220
which is where your actual files live

463
00:31:46,540 --> 00:31:50,710
and you have the $GIT_DIR, where the actual data lives

464
00:31:50,710 --> 00:31:55,900
normaly in a normal checkout you just have your directory and .git under this

465
00:31:57,430 --> 00:32:01,500
If you have a bare repository you obviously don't have an actual checkout of your data

466
00:32:01,500 --> 00:32:05,530
you have just all the objects and the configuration stuff

467
00:32:05,530 --> 00:32:09,190
so that's what a bare repository boils down to being

468
00:32:09,650 --> 00:32:12,710
A fake bare git repository on the other hand has both

469
00:32:12,710 --> 00:32:15,120
it has a $GIT_WORK_TREE and it has a $GIT_DIR

470
00:32:15,120 --> 00:32:16,920
but those are detached from each other

471
00:32:16,920 --> 00:32:19,520
they don't have to be closely tied together

472
00:32:19,890 --> 00:32:26,460
and also sets core.bare = false, to actually tell git that "yes, this is a real setup, but..."

473
00:32:26,460 --> 00:32:31,460
"yes, you still have a work tree, even thought you don't really expect it"

474
00:32:31,460 --> 00:32:33,460
"to have one, you still have a work tree".

475
00:32:34,800 --> 00:32:38,460
By default vcsh puts your work tree into home

476
00:32:38,460 --> 00:32:40,080
and your git dir into...

477
00:32:40,080 --> 00:32:45,080
it's based on .config/vcsh/repo.d and then the name of the repository

478
00:32:45,080 --> 00:32:49,800
which just puts it away and out the way of you actually seeing stuff

479
00:32:49,800 --> 00:32:54,670
but it follows the cross desktop specifications so if you move stuff around it will also follow

480
00:32:55,130 --> 00:32:57,030
Fake bare repositories are really...

481
00:32:58,190 --> 00:33:01,890
are messy to setup, and it's very easy to get them wrong

482
00:33:02,030 --> 00:33:07,360
that is also the reason why git normally disallows this kind of stuff

483
00:33:07,590 --> 00:33:10,000
because all of a sudden you have a lot of...

484
00:33:10,000 --> 00:33:12,590
context-dependency on when you do what

485
00:33:12,590 --> 00:33:14,590
just immagine you set git workdir...

486
00:33:14,860 --> 00:33:16,340
$GIT_WORK_DIR, sorry

487
00:33:16,340 --> 00:33:19,580
and run random commands like git add, that's...

488
00:33:19,580 --> 00:33:25,740
kind of ok, if you git reset --hard you'll probably not be to happy

489
00:33:25,740 --> 00:33:28,800
you checkout the current version that's also quite bad

490
00:33:28,800 --> 00:33:32,410
and if you clean -f, yeah, you just throw the home directory

491
00:33:32,410 --> 00:33:33,620
congratulations

492
00:33:33,620 --> 00:33:39,030
So, it's really risky to run with these variables set

493
00:33:39,360 --> 00:33:43,760
which is why I wrote vcsh to wrap around git

494
00:33:44,130 --> 00:33:49,690
to hide all this complexity and do quite some sanity checks to make sure everything's set up correctly

495
00:33:50,060 --> 00:33:57,470
again it allows you to have several repositories and it also manages really the complete lifecycle of all your repositories

496
00:33:57,470 --> 00:34:02,700
it's very easy to just create a new repository, you just init, just with git

497
00:34:02,700 --> 00:34:08,350
you add stuff, you commit it, and you define a remote and start pushing to this remote

498
00:34:08,670 --> 00:34:09,969
simple

499
00:34:11,219 --> 00:34:14,370
This looks like git because it's very closely tight to git

500
00:34:14,370 --> 00:34:19,190
and it uses a lot of the power or of the syntax of git, for obvious reasons

501
00:34:19,280 --> 00:34:21,969
because... it's closely tight to git

502
00:34:21,969 --> 00:34:25,250
you can simply clone as you would with git

503
00:34:25,250 --> 00:34:27,889
you can simply show your files as you would with git

504
00:34:27,889 --> 00:34:31,690
you can rename the repository, which git can't do, but you don't have to

505
00:34:31,690 --> 00:34:34,239
you can show the status of all your files

506
00:34:34,239 --> 00:34:36,239
or just of one of your repositories

507
00:34:36,239 --> 00:34:37,710
or of all repositories

508
00:34:37,900 --> 00:34:44,060
you can pull in all your repositories at once, you can push all of your repositories at once

509
00:34:44,060 --> 00:34:45,540
with one single command

510
00:34:47,110 --> 00:34:51,929
so, if you are on the road, or you just want to sync up a new machine it's really quick, it's really easy

511
00:34:53,460 --> 00:34:57,160
There are three modes of dealing with your repositories

512
00:34:57,160 --> 00:34:59,020
default mode is the quickest to type

513
00:34:59,020 --> 00:35:04,300
you just say vcsh zsh commit whatever or any random git command

514
00:35:04,530 --> 00:35:06,300
but you cannot really run gitk

515
00:35:06,300 --> 00:35:09,670
you can do this by using the run mode, which is the second mode

516
00:35:09,670 --> 00:35:14,070
we simply ? here run is missing and here git is missing

517
00:35:14,070 --> 00:35:18,700
so you say simply vcsh run zsh git commit whatever

518
00:35:18,700 --> 00:35:25,880
and this is exactly the same command, it's literally the same comand once it arrives at the shell level, so to speak

519
00:35:25,880 --> 00:35:28,560
here you can also run gitk, because...

520
00:35:28,560 --> 00:35:33,840
with this, you set up the whole environment for one single command to run with this context

521
00:35:33,840 --> 00:35:37,180
of the changed environment variables

522
00:35:37,180 --> 00:35:41,720
or you could even enter the repository, then you set all the variables

523
00:35:42,180 --> 00:35:45,790
and then you can just use normal git commands as you would normally

524
00:35:45,790 --> 00:35:47,510
this is the most powerful mode,

525
00:35:47,510 --> 00:35:51,720
but it's also the most likely to hurt you if you don't know what you're doing

526
00:35:51,720 --> 00:35:54,590
so I don't recommend working ? down this way.

527
00:35:56,580 --> 00:36:03,620
You should have your shell display prompt information about being in a vcsh repository or not

528
00:36:04,320 --> 00:36:08,020
simply because else you may forget that you entered something

529
00:36:08,020 --> 00:36:13,630
and then if you run those commands, there will be pain

530
00:36:17,700 --> 00:36:22,330
At once the usecases, which will be possible quite soon

531
00:36:22,330 --> 00:36:28,580
we can just combine vcsh with git-annex to manage everything which is not configuration files in your own home directory

532
00:36:28,580 --> 00:36:35,390
? basically two programs to sync everything about all of your home directory

533
00:36:35,390 --> 00:36:37,390
without having to do any extra work

534
00:36:38,220 --> 00:36:40,810
you can also use it to do really wierd stuff

535
00:36:40,810 --> 00:36:46,320
for example you can backup a .git of a different repository with the help of vcsh

536
00:36:46,320 --> 00:36:51,740
so you can just go in, change objects or anything, break stuff and just replay whatever you're doing

537
00:36:51,740 --> 00:36:55,720
just to try and see how it breaks in interesting ways.

538
00:36:56,000 --> 00:37:01,880
You can just backup a working copy which is maintained by a different reopository or a different system

539
00:37:02,160 --> 00:37:06,980
you can even put a whole repository, including the .git,

540
00:37:06,980 --> 00:37:08,460
into a different git file

541
00:37:08,460 --> 00:37:13,140
or you can even put other VCSs like subversion or something into git, if you want to.

542
00:37:14,300 --> 00:37:15,590
Then there is mr.

543
00:37:15,590 --> 00:37:17,680
mr ties all those...

544
00:37:17,680 --> 00:37:22,680
hopefully by now you have about twenty new repositories

545
00:37:22,680 --> 00:37:25,780
because you have configuration, you have ikiwiki, you have everything

546
00:37:25,920 --> 00:37:28,980
so now you need something to syncronize all those repositories

547
00:37:28,980 --> 00:37:32,400
because doing it by hand is just a lot of work

548
00:37:34,770 --> 00:37:40,970
mr supports push, pull, commit operations for all the major known version control systems

549
00:37:40,970 --> 00:37:44,630
allowing you to have one single interface to operate on all your systems

550
00:37:44,950 --> 00:37:48,570
It's quite trivial to write support for new systems

551
00:37:48,570 --> 00:37:52,320
I think it took me about two hours to support vcsh natively

552
00:37:52,320 --> 00:37:54,320
so, that's really quick

553
00:37:54,320 --> 00:37:57,040
If you want to try, the stuff which I told you about...

554
00:37:57,040 --> 00:38:04,730
in the links later there will be the possibility to just clone a subrepository for vcsh

555
00:38:04,960 --> 00:38:09,920
which will then put up a suggested mr directory layout

556
00:38:09,920 --> 00:38:11,920
and you can just work from there

557
00:38:11,920 --> 00:38:16,220
This is the... alright, it's my suggested layout

558
00:38:16,220 --> 00:38:17,510
which basically...

559
00:38:17,880 --> 00:38:21,680
you just include everything in config.d you maintain...

560
00:38:21,680 --> 00:38:30,340
your available.d, by means of vcsh, so you simply sync around all your content between all the different computers

561
00:38:30,340 --> 00:38:35,020
and then you simply soft link from available to the actual config

562
00:38:35,020 --> 00:38:39,370
which is basically what apache does with sites.enabled and sites.available

563
00:38:39,370 --> 00:38:42,660
or modules.available and modules.enabled

564
00:38:42,660 --> 00:38:44,660
which is really really powerful

565
00:38:45,020 --> 00:38:48,400
Last thing is not git based, but zsh.

566
00:38:48,630 --> 00:38:51,690
It's a really powerful shell, you should consider using it

567
00:38:51,690 --> 00:38:56,000
it has very good tab complection for all the tools listed here, more than bash

568
00:38:56,000 --> 00:39:00,030
it has a right prompt, which will automatically disappear if it needs to

569
00:39:00,030 --> 00:39:04,840
which is very convenient to display not important but still useful information

570
00:39:04,840 --> 00:39:11,050
and it will automatically, if you tell it to, tell you about you being in a git repository or subversion repository or whatever

571
00:39:11,050 --> 00:39:12,300
by means of vcs.info

572
00:39:12,810 --> 00:39:17,670
which also means you'll be told that at the moment you are in a vcsh repository

573
00:39:17,670 --> 00:39:20,960
and you may kill your stuff if you do things wrong

574
00:39:20,960 --> 00:39:22,960
it can mimic all the major shells

575
00:39:22,960 --> 00:39:25,960
and there's just too many reasons to list

576
00:39:26,980 --> 00:39:28,830
So... final pitch

577
00:39:28,830 --> 00:39:34,300
This is true: I've tried it earlier, I can demo it, I still have five minutes left

578
00:39:34,300 --> 00:39:38,930
it takes me less than five minutes to syncronize my complete, whole, digital life while on the road

579
00:39:38,930 --> 00:39:44,260
so if I'm at the airport and just want to update all my stuff,and push out all my stuff...

580
00:39:44,260 --> 00:39:47,270
it'll take a few minutes, but then I can hop on the airplane...

581
00:39:47,270 --> 00:39:51,060
and I'll know everything is fine, everything is up-to-date on my local machine

582
00:39:51,060 --> 00:39:56,620
on my laptop machine, I can continue working, and have a backup on my remote systems

583
00:39:57,040 --> 00:39:59,210
These are the websites

584
00:39:59,770 --> 00:40:08,010
The slides will be linked from penta, so you are more than welcome to look at these links later

585
00:40:08,010 --> 00:40:12,040
There are previous talks, which you can also look at, if you want to

586
00:40:12,040 --> 00:40:13,710
and that's pretty much it

587
00:40:13,710 --> 00:40:16,910
and if you have any more questions afterwards either catch me...

588
00:40:16,910 --> 00:40:20,930
or there is an IRC channel, and there is a mailing list

589
00:40:20,930 --> 00:40:26,770
ok, we can take a few questions, we have still a few minutes

590
00:40:26,770 --> 00:40:31,170
then if there are more questions ask Ritchie afterwards

591
00:40:31,820 --> 00:40:36,360
And while we are doing this just look here, because that's a complete sync of everything I have

592
00:40:37,330 --> 00:40:39,780
Just to make sure I understood this correctly,

593
00:40:39,780 --> 00:40:48,680
with git-annex the point is that the data is stored dispersed over different local destinations, so to speak

594
00:40:48,680 --> 00:40:53,210
but the metadata ? exists, ? complete git history

595
00:40:53,210 --> 00:40:59,880
so git is able to tell me, "well, this version at that destination was changed at that time and so on and so on"

596
00:41:00,390 --> 00:41:02,800
did I get this right or...

597
00:41:02,800 --> 00:41:05,440
git will be able to tell you about changes...

598
00:41:05,440 --> 00:41:07,760
ok, I don't have internet, sorry

599
00:41:07,760 --> 00:41:11,830
git will be able to tell you about changes in the filenames, or directory structure

600
00:41:11,830 --> 00:41:15,910
git-annex will be able to tell you about changes in the actual file content

601
00:41:15,910 --> 00:41:17,440
or in moving around the files

602
00:41:17,440 --> 00:41:21,650
but as one single unit, more or less, then yes...

603
00:41:22,070 --> 00:41:24,800
the answer is yes, but not quite, but yes

604
00:41:24,800 --> 00:41:32,350
yes, but ? all the things you asked about are in git, you know the previous location, all that stuff

605
00:41:32,350 --> 00:41:39,110
but in a separate branch which you should use git-annex to access, but you can do it by hand if you want to

606
00:41:51,570 --> 00:41:54,530
I'm not familiar with tracking branches,

607
00:41:54,530 --> 00:41:58,330
yet you mention that the workflow for your wife has a different view of the data than you

608
00:41:58,330 --> 00:42:06,670
with that workflow is it possible for your wife to upload photos that you will have in your view as well, or is it a oneway street?

609
00:42:07,220 --> 00:42:12,730
minor correction: tracking branches track a different repository,

610
00:42:12,730 --> 00:42:17,500
what I meant was rebasing branches, which rebase on top of a different branch

611
00:42:17,500 --> 00:42:23,890
which basically just keeps the patches always on top of the branch, no matter where the head moves to

612
00:42:26,580 --> 00:42:32,180
if she wanted to do that she would need to simply git checkout master

613
00:42:32,180 --> 00:42:38,620
do whatever she wanted to do, and then git checkout her own branch, and then she's...

614
00:42:38,620 --> 00:42:44,460
she is able to, but she would need to change into the master branch and then back

615
00:42:49,090 --> 00:42:50,110
microphone

616
00:42:51,030 --> 00:42:56,640
she never pushes her private branch? it always lives on her own machine?

617
00:42:56,640 --> 00:43:01,690
no, she does push it, but I don't display this view of the data

618
00:43:02,700 --> 00:43:07,520
because else she wouldn't be able to syncronize this view between different computers

619
00:43:08,080 --> 00:43:11,780
I seem to have internet now, so let's just let this run in the background

620
00:43:13,680 --> 00:43:15,020
any more questions?

621
00:43:23,360 --> 00:43:24,560
no more questions?

622
00:43:26,690 --> 00:43:27,440
than we...

623
00:43:27,440 --> 00:43:29,440
? more minutes for questions?

624
00:43:36,230 --> 00:43:41,280
ok, so thanks to Richard Hartmann, we will continue...