1
00:00:00,000 --> 00:00:15,010
*34c3 intro*

2
00:00:15,010 --> 00:00:18,339
Herald: ... used Anja Dahlmann, a
political scientist and researcher at

3
00:00:18,339 --> 00:00:23,310
Stiftung Wissenschaft und Politik, a
berlin-based think-tank. Here we go.

4
00:00:23,310 --> 00:00:30,160
*applause*

5
00:00:30,160 --> 00:00:40,030
Anja Dahlmann: Yeah, Thanks for being
here. I probably neither cut myself nor

6
00:00:40,030 --> 00:00:44,270
proposed but I hope it's still
interesting. I'm going to talk about

7
00:00:44,270 --> 00:00:49,240
preventive arms control and international
humanitarian law and doing in this

8
00:00:49,240 --> 00:00:53,270
international debate around autonomous
weapons. This type of weapon is also

9
00:00:53,270 --> 00:00:59,010
referred to as Lethal Autonomous Weapons
System, short LAWS, or also killer robots.

10
00:00:59,010 --> 00:01:04,239
So if I say LAWS, I mostly mean these
weapons and not like legal laws, just to

11
00:01:04,239 --> 00:01:11,650
confuse you a bit. Okay. I will discuss
this topic along three questions. First of

12
00:01:11,650 --> 00:01:18,300
all, what are we actually talking about
here, what are autonomous weapons? Second,

13
00:01:18,300 --> 00:01:22,210
why should we even care about this? Why's
it important? And third, how could this

14
00:01:22,210 --> 00:01:31,480
issue be addressed on international level?
So. I'll go through my slides, anyway,

15
00:01:31,480 --> 00:01:37,770
what are we talking about here? Well,
during the international negotiations, so

16
00:01:37,770 --> 00:01:45,360
far no real, no common definition has been
found. So States, Parties try to find

17
00:01:45,360 --> 00:01:50,090
something or not and for my presentation I
will just use a very broad definition of

18
00:01:50,090 --> 00:01:57,229
autonomous weapons, which is: Weapons that
can once activated execute a broad range

19
00:01:57,229 --> 00:02:02,350
of tasks or selecting to engage targets
without further human intervention. And

20
00:02:02,350 --> 00:02:08,340
it's just a very broad spectrum of weapons
that might fall under this definition.

21
00:02:08,340 --> 00:02:13,150
Actually, some existing ones are there as
well which you can't see here. That would

22
00:02:13,150 --> 00:02:19,940
be the Phalanx system for example. It's
been around since the 1970s. Sorry...

23
00:02:19,940 --> 00:02:22,950
Herald: Man kann nichts hören
auf der Bühne. Mach mal weiter.

24
00:02:22,950 --> 00:02:27,310
Dahlmann: Sorry. So, Phalanx system has
been around since the 1970s, a US system,

25
00:02:27,310 --> 00:02:33,069
air defense system, based on ships and
it's been to - just yeah, defend the ship

26
00:02:33,069 --> 00:02:39,330
against incoming objects from the air. So
that's around, has been around for quite a

27
00:02:39,330 --> 00:02:45,170
long time and it might be even part of
this LAWS definition or not but just to

28
00:02:45,170 --> 00:02:49,410
give you an impression how broad this
range is: Today, we've got for example

29
00:02:49,410 --> 00:02:57,990
demonstrators like the Taranis drone, a UK
system, or the x74b which can, for

30
00:02:57,990 --> 00:03:04,959
example, autonomously land
*applause*

31
00:03:04,959 --> 00:03:08,700
land on aircraft carriers and can be air-
refueled and stuff like that which is

32
00:03:08,700 --> 00:03:13,550
apparently quite impressive if you don't
need a human to do that and in the future

33
00:03:13,550 --> 00:03:18,810
there might be even, or there probably
will be even more, autonomous functions,

34
00:03:18,810 --> 00:03:25,650
so navigation, landing, refueling, all
that stuff. That's, you know, old but at

35
00:03:25,650 --> 00:03:30,090
some point there might, be weapons might
be able to choose their own ammunition

36
00:03:30,090 --> 00:03:34,560
according to the situation. They might be
able to choose their target and decide

37
00:03:34,560 --> 00:03:41,720
when to engage with the target without any
human intervention at some point. And

38
00:03:41,720 --> 00:03:45,001
that's quite problematic, I will tell you
why that's in a minute. Overall, you can

39
00:03:45,001 --> 00:03:52,230
see that there's a gradual decline of
human control over weapons systems or over

40
00:03:52,230 --> 00:03:58,050
weapons and the use of force. So that's a
very short and broad impression of what

41
00:03:58,050 --> 00:04:01,360
we're talking about here. And talking
about definitions, it's always interesting

42
00:04:01,360 --> 00:04:05,730
what you're not talking about and that's
why I want to address some misconceptions

43
00:04:05,730 --> 00:04:13,481
in the public debate. First of all, when
we talk about machine autonomy, also

44
00:04:13,481 --> 00:04:19,858
artificial intelligence, with intelligence
which is the technology behind this,

45
00:04:19,858 --> 00:04:25,420
people - not you probably - in the media
and the broader public often get the idea

46
00:04:25,420 --> 00:04:30,540
that these machines might have some kind
of real intelligence or intention or an

47
00:04:30,540 --> 00:04:36,260
entity on own right and they're just not.
It's just statistical methods, it's just

48
00:04:36,260 --> 00:04:40,740
math and you know way more about this than
I do so I will leave it with this and just

49
00:04:40,740 --> 00:04:45,600
say that or highlight that they have these
machines, these weapons have certain

50
00:04:45,600 --> 00:04:50,070
competences for specific tasks. They are
not entities on their own right, they are

51
00:04:50,070 --> 00:04:55,050
not intentional.And that's important when
we talk about ethical and legal challenges

52
00:04:55,050 --> 00:05:06,580
afterwards. Sorry. There it is. And the
other, in connection with this, there's

53
00:05:06,580 --> 00:05:11,380
another one, which is the plethora of
Terminator references in the media as soon

54
00:05:11,380 --> 00:05:15,000
as you talk about autonomous weapons,
mostly referred to as killer robots in

55
00:05:15,000 --> 00:05:20,251
this context. And just in case you tend to
write an article about this: don't use a

56
00:05:20,251 --> 00:05:24,340
Terminator picture, please. Don't, because
it's really unhelpful to understand where

57
00:05:24,340 --> 00:05:30,350
the problems are. With this kind of thing,
people assume that we have problems is

58
00:05:30,350 --> 00:05:34,160
when we have machines with a human-like
intelligence which look like the

59
00:05:34,160 --> 00:05:39,750
Terminator or something like this. And the
problem is that really way before that

60
00:05:39,750 --> 00:05:46,820
they start when you use assisting systems
when you have men or human-machine teaming

61
00:05:46,820 --> 00:05:50,860
or when you accumulate a couple of
autonomous functions through the targeting

62
00:05:50,860 --> 00:05:57,720
cycle. So through this, the military steps
are lead to the use of force or lead to

63
00:05:57,720 --> 00:06:03,560
the killing of people. And that's not,
this is really not our problem at the

64
00:06:03,560 --> 00:06:08,350
moment. So please keep this in mind
because it's not just semantics, semantics

65
00:06:08,350 --> 00:06:15,350
to differentiate between these two things.
It's really manages the expectations of

66
00:06:15,350 --> 00:06:20,780
political and military decision-makers.
Ok, so now you've got kind of an

67
00:06:20,780 --> 00:06:23,620
impression what I'm talking about here so
why should we actually talk about this?

68
00:06:23,620 --> 00:06:30,090
What's all the fuss about? Actually,
autonomous weapons have or would have

69
00:06:30,090 --> 00:06:34,169
quite a few military advantages: They
might be, in some cases, faster or even

70
00:06:34,169 --> 00:06:39,780
more precise than humans. And you don't
need a constant communication link. So you

71
00:06:39,780 --> 00:06:43,740
don't have, you don't have to worry about
instable communication links, you don't

72
00:06:43,740 --> 00:06:50,020
have to worry about latency or detection
or a vulnerability of this specific link.

73
00:06:50,020 --> 00:06:57,050
So yay! And a lot of, let's say very
interesting, military options come from

74
00:06:57,050 --> 00:07:02,520
that. People talk about stealthy
operations and shallow waters for example.

75
00:07:02,520 --> 00:07:07,340
Or you know remote missions and secluded
areas, things like that. And you can get

76
00:07:07,340 --> 00:07:13,830
very creative with tiniest robots and
swarms for example. So shiny new options.

77
00:07:13,830 --> 00:07:19,630
But, and of course there's a "but", it
comes at a prize because you have at least

78
00:07:19,630 --> 00:07:26,590
three dimensions of challenges in this
regard. First of all, the legal ones. When

79
00:07:26,590 --> 00:07:31,389
we talk about these weapons, they might
be, they will be applied in conflict where

80
00:07:31,389 --> 00:07:37,500
international humanitarian law IHL
applies. And IHL consists of quite a few

81
00:07:37,500 --> 00:07:45,430
very abstract principles. For example:
principle of distinction between

82
00:07:45,430 --> 00:07:51,310
combatants and civilians, principle of
proportionality or a military necessity.

83
00:07:51,310 --> 00:07:58,069
They are very abstract and I'm pretty sure
they really always need a human judgment

84
00:07:58,069 --> 00:08:05,889
to interpret this, these principles, and
apply them to dynamic situations. Feel

85
00:08:05,889 --> 00:08:14,870
free to correct me if I'm wrong later. So
that's one thing. So if you remove the

86
00:08:14,870 --> 00:08:19,440
human from the targeting cycle, this human
judgment might be missing and therefore

87
00:08:19,440 --> 00:08:24,700
military decision makers have to evaluate
very carefully the quality of human

88
00:08:24,700 --> 00:08:31,699
control and human judgement within the
targeting cycle. So that's law. Second

89
00:08:31,699 --> 00:08:38,719
dimension of challenges are security
issues. When you look at these new systems

90
00:08:38,719 --> 00:08:43,639
they are cool and shiny and as most new
types of weapons they are, they have the

91
00:08:43,639 --> 00:08:49,050
potential to stir an arms race between
between states. So they actually might

92
00:08:49,050 --> 00:08:53,559
make conflicts more likely just because
they are there and states want to have

93
00:08:53,559 --> 00:09:00,519
them and feel threatened by them. Second
aspect is proliferation. Autonomy is based

94
00:09:00,519 --> 00:09:04,739
on software, so software can be easily
transferred it's really hard to control

95
00:09:04,739 --> 00:09:08,579
and all the other components, or most of
the other components you will need, are

96
00:09:08,579 --> 00:09:12,279
available on the civilian market so you
can build this stuff on your own if you're

97
00:09:12,279 --> 00:09:19,639
smart enough. So we have might have more
conflicts from these types of weapons and

98
00:09:19,639 --> 00:09:24,500
it's might get, well, more difficult to
control the application of this

99
00:09:24,500 --> 00:09:29,560
technology. And the third one which is it
especially worrying for me is the as

100
00:09:29,560 --> 00:09:33,971
potential for escalation within the
conflict, especially when you have, when

101
00:09:33,971 --> 00:09:40,279
both or more sites use these autonomous
weapons, you have these very complex

102
00:09:40,279 --> 00:09:45,550
adversary systems and it will become very
hard to predict how they are going to

103
00:09:45,550 --> 00:09:52,050
interact. They will increase the speed of
the of the conflict and the human might

104
00:09:52,050 --> 00:09:57,040
not even have a chance to process
what's going on there.

105
00:09:57,040 --> 00:10:01,949
So that's really worrying and we can see
for example in high-frequency trading at

106
00:10:01,949 --> 00:10:05,980
the stock markets where problems arise
there and how are difficult is for humans

107
00:10:05,980 --> 00:10:12,529
to understand what's going on there. So
that, that are of some of these security

108
00:10:12,529 --> 00:10:23,129
issues there. And the last and maybe maybe
most important one are ethics. As I

109
00:10:23,129 --> 00:10:29,089
mentioned before, when you use autonomy
and weapons or machines you have

110
00:10:29,089 --> 00:10:32,759
artificial intelligence so you don't have
real intention, a real entity that's

111
00:10:32,759 --> 00:10:38,220
behind this. So the killing decision might
at some point be based on statistical

112
00:10:38,220 --> 00:10:43,209
methods and no one will be involved there
and that's, well, worrying for a lot of

113
00:10:43,209 --> 00:10:48,399
reasons but also it could constitute a
violation of human dignity. You can argue

114
00:10:48,399 --> 00:10:54,079
that humans have, well, you can kill
humans in in war but they at least have

115
00:10:54,079 --> 00:10:59,140
the right to be killed by another human or
at least by the decision of another human,

116
00:10:59,140 --> 00:11:02,629
but we can discuss this later.
So at least on this regard it would be

117
00:11:02,629 --> 00:11:07,610
highly unethical and that really just
scratches the surface of problems and

118
00:11:07,610 --> 00:11:12,790
challenges that would arise from the use
of these autonomous weapons. I haven't

119
00:11:12,790 --> 00:11:16,680
even touched on the problems with training
data, with accountability, with

120
00:11:16,680 --> 00:11:23,299
verification and all that funny stuff
because I only have 20 minutes. So, sounds

121
00:11:23,299 --> 00:11:32,519
pretty bad, doesn't it? So how can this
issue be addressed? Luckily, states have,

122
00:11:32,519 --> 00:11:36,739
thanks to a huge campaign of NGOs, noticed
that there might be some problems and

123
00:11:36,739 --> 00:11:40,439
there might be a necessity to address
that, this issue. They're currently doing

124
00:11:40,439 --> 00:11:45,379
this in the UN Convention on certain
conventional weapons, CCW, where they

125
00:11:45,379 --> 00:11:53,290
discuss a potential ban of the development
and use of these lethal weapons or weapons

126
00:11:53,290 --> 00:11:57,589
that lack meaningful human control over
the use of force. There are several ideas

127
00:11:57,589 --> 00:12:04,420
around there. And such a ban would be
really the maximum goal of the NGOs there

128
00:12:04,420 --> 00:12:09,170
but it becomes increasingly unlikely that
this happens. Most states do not agree

129
00:12:09,170 --> 00:12:12,991
with a complete ban, they want to regulate
it a bit here, a bit there, and they

130
00:12:12,991 --> 00:12:17,879
really can't find a common common
definition as I mentioned before because

131
00:12:17,879 --> 00:12:22,609
if you have a broad definition as just as
I used it you will notice that you have

132
00:12:22,609 --> 00:12:26,009
existing systems in there that might be
not that problematic or that you just

133
00:12:26,009 --> 00:12:31,749
don't want to ben and you might stop
civilian or commercial developments which

134
00:12:31,749 --> 00:12:38,690
you also don't want to do. So states are
stuck on this regard and they also really

135
00:12:38,690 --> 00:12:42,179
challenge the notion that we need a
preventive arms control here, so that we

136
00:12:42,179 --> 00:12:50,629
need to act before these systems are
applied on the battlefield. So at the

137
00:12:50,629 --> 00:12:55,980
moment, this is the fourth year or
something of these negotiations and we

138
00:12:55,980 --> 00:13:00,089
will see how it goes this year and if
states can't find a common ground there it

139
00:13:00,089 --> 00:13:04,949
becomes increasingly like or yeah becomes
likely that it will change to another

140
00:13:04,949 --> 00:13:11,060
forum just like with anti-personnel mines
for example which where the the treaty was

141
00:13:11,060 --> 00:13:17,639
found outside of the United Nations. But
yeah, the window of opportunity really

142
00:13:17,639 --> 00:13:24,950
closes and states and NGOs have to act
there and yeah keep on track there. Just

143
00:13:24,950 --> 00:13:32,339
as a side note, probably quite a few
people are members of NGOs so if you look

144
00:13:32,339 --> 00:13:39,059
at the campaign to stop killer robots with
a big campaign behind this, this process,

145
00:13:39,059 --> 00:13:43,310
there's only one German NGO which is
facing finance, so if you're especially if

146
00:13:43,310 --> 00:13:48,739
you're German NGO and are interest that in
AI it might be worthwhile to look into the

147
00:13:48,739 --> 00:13:54,509
military dimension as well. We really need
some expertise on that regard, especially

148
00:13:54,509 --> 00:14:00,439
on AI and these technologies. They're...
Okay, so just in case you fell asleep in

149
00:14:00,439 --> 00:14:05,589
the last 15 minutes I want you to take
away three key messages: Please be aware

150
00:14:05,589 --> 00:14:11,100
of the trends and internal logic that lead
to autonomy in weapons. Do not

151
00:14:11,100 --> 00:14:15,629
overestimate the abilities of autonomy, of
autonomous machines like intent and these

152
00:14:15,629 --> 00:14:20,189
things and because you probably all knew
this already, please tell people about

153
00:14:20,189 --> 00:14:23,899
this, tell other people about this,
educate them about this type of

154
00:14:23,899 --> 00:14:30,669
technology. And third, don't underestimate
the potential dangers for security and

155
00:14:30,669 --> 00:14:37,249
human dignity that comes from this type of
weapon. I hope that I could interest you a

156
00:14:37,249 --> 00:14:40,839
bit more in this in this particular issue
if you want to learn more you can find

157
00:14:40,839 --> 00:14:46,819
really interesting sources on the website
of the CCW at the campaign to stuff killer

158
00:14:46,819 --> 00:14:53,410
robots and from a research project that I
happen to work in, the International Panel

159
00:14:53,410 --> 00:14:56,759
on the Regulation of Autonomous Weapons,
we do have a few studies on that regard

160
00:14:56,759 --> 00:15:02,059
and we're going to publish a few more. So
please, check this out and thank you for

161
00:15:02,059 --> 00:15:03,481
your attention.

162
00:15:03,481 --> 00:15:13,714
*Applause*

163
00:15:13,714 --> 00:15:16,199
Questions?

164
00:15:20,519 --> 00:15:23,779
Herald: Sorry. So we have some time for
questions answers now. Okay, first of all

165
00:15:23,779 --> 00:15:28,110
I have to apologize that we had a hiccup
with the signing language, the acoustics

166
00:15:28,110 --> 00:15:32,209
over here on the stage was so bad that she
didn't could do her job so I'm

167
00:15:32,209 --> 00:15:38,720
terrible sorry about that. We fixed it in
the talk and my apologies for that. We are

168
00:15:38,720 --> 00:15:42,160
queuing here on the microphones already so
we start with microphone number one, your

169
00:15:42,160 --> 00:15:44,610
question please.
Mic 1: Thanks for your talk Anja. Don't

170
00:15:44,610 --> 00:15:49,049
you think there is a possibility to reduce
war crimes as well by taking away the

171
00:15:49,049 --> 00:15:54,059
decision from humans and by having
algorithms who decide which are actually

172
00:15:54,059 --> 00:15:56,929
auditable?
Dahlmann: Yeah that's, actually, that's

173
00:15:56,929 --> 00:15:59,740
something I just discussed in the
international debate as well, that there

174
00:15:59,740 --> 00:16:05,399
might, that machines might be more ethical
than humans could be. And well, of course

175
00:16:05,399 --> 00:16:11,509
they won't just start raping women because
they want to but you can program them to

176
00:16:11,509 --> 00:16:18,269
do this. So you just you shift the
problems really. And also maybe these

177
00:16:18,269 --> 00:16:22,559
machines don't get angry but they don't
show compassion either so if you are there

178
00:16:22,559 --> 00:16:26,069
and your potential target they just won't
stop they will just kill you and do not

179
00:16:26,069 --> 00:16:32,839
think once think about this. So you have
to really look at both sides there I guess.

180
00:16:32,839 --> 00:16:37,839
Herald: Thanks. So we switch over
to microphone 3, please.

181
00:16:37,839 --> 00:16:44,699
Mic 3: Thanks for the talk. Regarding
autonomous cars, self-driving cars,

182
00:16:44,699 --> 00:16:49,239
there's a similar discussion going on
regarding the ethics. How should a car

183
00:16:49,239 --> 00:16:54,139
react in a case of an accident? Should it
protect people outside people, inside,

184
00:16:54,139 --> 00:17:02,069
what are the laws? So there is another
discussion there. Do you work with people

185
00:17:02,069 --> 00:17:07,270
in this area or is this is there any
collaboration?

186
00:17:07,270 --> 00:17:10,169
Dahlmann: Maybe there's less collaboration
than one might think there is. I think

187
00:17:10,169 --> 00:17:17,299
there is. Of course, we we monitor this
debate as well and yeah we think about the

188
00:17:17,299 --> 00:17:20,689
possible applications of the outcomes for
example from this German ethical

189
00:17:20,689 --> 00:17:26,849
commission on self-driving cars for our
work. But I'm a bit torn there because

190
00:17:26,849 --> 00:17:30,910
when you talk about weapons, they are
designed to kill people and cars mostly

191
00:17:30,910 --> 00:17:36,000
are not. So with this ethical committee
you want to avoid killing people or decide

192
00:17:36,000 --> 00:17:42,100
what happens when this accident occurs. So
they are a bit different but of course

193
00:17:42,100 --> 00:17:48,790
yeah you can learn a lot from both
discussions and we aware of that.

194
00:17:48,790 --> 00:17:53,530
Herald: Thanks. Then we're gonna go over
in the back, microphone number 2, please.

195
00:17:53,530 --> 00:17:59,500
Mic 2: Also from me thanks again for this
talk and infusing all this professionalism

196
00:17:59,500 --> 00:18:10,280
into the debate because some of the
surroundings of our, so to say ours

197
00:18:10,280 --> 00:18:17,860
scenery, they like to protest against very
specific things like for example the

198
00:18:17,860 --> 00:18:23,920
Rammstein air base and in my view that's a
bit misguided if you just go out and

199
00:18:23,920 --> 00:18:30,559
protest in a populistic way without
involving these points of expertise that

200
00:18:30,559 --> 00:18:38,210
you offer. And so, thanks again for that.
And then my question: How would you

201
00:18:38,210 --> 00:18:46,940
propose that protests progress and develop
themselves to a higher level to be on the

202
00:18:46,940 --> 00:18:55,170
one hand more effective and on the other
hand more considerate of what is at stake

203
00:18:55,170 --> 00:19:00,200
on all the levels and on
all sides involved?

204
00:19:00,200 --> 00:19:05,690
Dahlmann: Yeah well, first, the Rammstein
issue is completely, actually a completely

205
00:19:05,690 --> 00:19:10,340
different topic. It's drone warfare,
remotely piloted drones, so there are a

206
00:19:10,340 --> 00:19:14,290
lot of a lot of problems with this and
we're starting killings but it's not about

207
00:19:14,290 --> 00:19:21,820
lethal autonomous weapons in particular.
Well if you want to be a part of this

208
00:19:21,820 --> 00:19:25,330
international debate, there's of course
this campaign to stop killer robots and

209
00:19:25,330 --> 00:19:30,429
they have a lot of really good people and
a lot of resources, sources, literature

210
00:19:30,429 --> 00:19:35,190
and things like that to really educate
yourself what's going on there, so that

211
00:19:35,190 --> 00:19:39,490
would be a starting point. And then yeah
just keep talking to scientists about

212
00:19:39,490 --> 00:19:45,280
this and find out where we see the
problems and I mean it's always helpful

213
00:19:45,280 --> 00:19:52,969
for scientists to to talk to people in the
field, so to say. So yeah, keep talking.

214
00:19:52,969 --> 00:19:55,519
Herald: Thanks for that. And the
signal angel signaled that we have

215
00:19:55,519 --> 00:19:59,200
something from the internet.
Signal Angel: Thank you. Question from

216
00:19:59,200 --> 00:20:04,210
IRC: Aren't we already in a killer robot
world? The bot net can attack a nuclear

217
00:20:04,210 --> 00:20:08,270
power plant for example. What do you think?
Dahlmann: I really didn't understand a

218
00:20:08,270 --> 00:20:09,710
word, I'm sorry.
Herald: I didn't understand that as well,

219
00:20:09,710 --> 00:20:12,590
so can you speak closer to
the microphone, please?

220
00:20:12,590 --> 00:20:16,480
Signal Angel: Yes. Aren't we already in a
killer robot world?

221
00:20:16,480 --> 00:20:19,510
Herald: Sorry, that doesn't work. Sorry.
Sorry, we stop that here, we can't hear it

222
00:20:19,510 --> 00:20:21,550
over here. Sorry.
Signal Angel: Okay.

223
00:20:21,550 --> 00:20:25,749
Herald: We're gonna switch over to
microphone two now, please.

224
00:20:25,749 --> 00:20:32,529
Mic 2: I have one little question. So in
your talk, you were focusing on the

225
00:20:32,529 --> 00:20:38,970
ethical questions related to lethal
weapons. Are you aware of ongoing

226
00:20:38,970 --> 00:20:45,379
discussions regarding the ethical aspects
of the design and implementation of less

227
00:20:45,379 --> 00:20:52,769
than lethal autonomous weapons for crowd
control and similar purposes?

228
00:20:52,769 --> 00:20:57,039
Dahlmann: Yeah actually within the CCW,
every term of this Lethal Autonomous

229
00:20:57,039 --> 00:21:02,660
Weapon Systems is disputed also the
"lethal" aspect and for the regulation

230
00:21:02,660 --> 00:21:07,670
that might be easier to focus on this for
now because less than lethal weapons come

231
00:21:07,670 --> 00:21:14,490
with their own problems and the question
if they are ethical and if they can, if

232
00:21:14,490 --> 00:21:18,820
IHL applies to them but I'm not really
deep into this discussion. So I'll just

233
00:21:18,820 --> 00:21:22,600
have to leave it there.
Herald: Thanks and back here to microphone

234
00:21:22,600 --> 00:21:25,230
one, please.
Mic 1: Hi. Thank you for the talk very

235
00:21:25,230 --> 00:21:31,710
much. My question is in the context of the
decreasing cost of both, the hardware and

236
00:21:31,710 --> 00:21:37,190
software, over the next say 20, 40 years.
Outside of a nation-state context like

237
00:21:37,190 --> 00:21:42,419
private forces or non nation-state actors
gaining use of these weapons, do things

238
00:21:42,419 --> 00:21:45,860
like the UN convention or the campaign to
stop killer robots apply are they

239
00:21:45,860 --> 00:21:52,409
considering private individuals trying to
leverage these against others?

240
00:21:52,409 --> 00:21:59,860
Dahlmann: Not sure what the campaign says
about this, I'm not a member there. The

241
00:21:59,860 --> 00:22:06,600
the CCW mostly focuses on international
humanitarian law which is important but I

242
00:22:06,600 --> 00:22:10,820
think it's it's not broad enough. So
questions like proliferation and all this

243
00:22:10,820 --> 00:22:16,480
is connected to your question and not
really or probably won't be part of

244
00:22:16,480 --> 00:22:21,090
regulation there. It's discussed only on
the edges of the of the debates and

245
00:22:21,090 --> 00:22:26,799
negotiations there but it doesn't seem to
be a really issue there.

246
00:22:26,799 --> 00:22:29,860
Mic 1: Thanks.
Herald: And over to microphone six,

247
00:22:29,860 --> 00:22:32,510
please.
Mic 6: Thank you. I have a question as a

248
00:22:32,510 --> 00:22:38,559
researcher: Do you know how far the
development has gone already? So how

249
00:22:38,559 --> 00:22:44,770
transparent or intransparent is your look
into what is being developed and

250
00:22:44,770 --> 00:22:50,669
researched on the side of militaria
working, military people working with

251
00:22:50,669 --> 00:22:54,789
autonomous weapons and developing them?
Dahlmann: Well, for me it's quite

252
00:22:54,789 --> 00:22:59,759
intransparent because I only have only
access to public publicly available

253
00:22:59,759 --> 00:23:04,630
sources so I don't really know what what's
going on behind closed doors in the

254
00:23:04,630 --> 00:23:10,500
military or in the industry there. Of
course you can you can monitor the

255
00:23:10,500 --> 00:23:14,820
civilian applications or developments
which can tell a lot about the the state

256
00:23:14,820 --> 00:23:24,131
of the art and for example the DARPA
the American Development Agency, they

257
00:23:24,131 --> 00:23:30,070
published sometimes a call for papers,
that's not the term, but there you can see

258
00:23:30,070 --> 00:23:34,440
where in which areas they are interested
in then for example they really like this

259
00:23:34,440 --> 00:23:40,990
idea of autonomous killer bug that can
act in swarms and monitor or even kill

260
00:23:40,990 --> 00:23:45,600
people and things like that. So yeah we
try to piece it, piece it together in

261
00:23:45,600 --> 00:23:48,699
our work.
Herald: We do have a little bit more time,

262
00:23:48,699 --> 00:23:50,570
are you okay to answer more questions?
Dahlmann: Sure.

263
00:23:50,570 --> 00:23:52,959
Herald: Then we're gonna switch over to
microphone three, please.

264
00:23:52,959 --> 00:24:00,460
Mic 3: Yes, hello. I think we are living
already in a world of Leathal Autonomous

265
00:24:00,460 --> 00:24:04,870
Weapon Systems if you think about these
millions of landmines which are operating.

266
00:24:04,870 --> 00:24:09,249
And so the question is: Shouldn't it be
possible to ban these weapon systems the

267
00:24:09,249 --> 00:24:14,320
same way as land mines that are already
banned by several countries so just

268
00:24:14,320 --> 00:24:19,240
include them in that definition? And
because the arguments should be very

269
00:24:19,240 --> 00:24:23,049
similar.
Dahlmann: Yeah it does, it does come to

270
00:24:23,049 --> 00:24:26,539
mind of course because these mines are
just lying around there and no one's

271
00:24:26,539 --> 00:24:32,900
interacting when you step on them and
boom! But they are, well it depends, it

272
00:24:32,900 --> 00:24:38,889
depends first of all a bit of your
definition of autonomy. So some say

273
00:24:38,889 --> 00:24:42,539
autonomous is when you act in dynamic
situations and the other ones would be

274
00:24:42,539 --> 00:24:47,889
automated and things like that and I think
this autonomy aspect, I really don't want

275
00:24:47,889 --> 00:24:55,899
to find, don't want to find define
autonomy here really but this this action

276
00:24:55,899 --> 00:25:01,480
in more dynamic spaces and the aspect of
machine learning and all these things,

277
00:25:01,480 --> 00:25:06,450
they are way more complex and they bring
different problems than just land mines.

278
00:25:06,450 --> 00:25:10,759
Landmines are problematic, anti-personnel
mines are banned for good reasons but they

279
00:25:10,759 --> 00:25:15,409
don't have the same problems I think. So
it won't be, I don't think it won't be

280
00:25:15,409 --> 00:25:21,539
sufficient to just put the LAWS in there,
the Lethal Autonomous Weapons.

281
00:25:21,539 --> 00:25:26,399
Herald: Thank you very much. I can't see
anyone else queuing up so therefore, Anja,

282
00:25:26,399 --> 00:25:28,990
thank you very much it's your applause!

283
00:25:28,990 --> 00:25:31,810
*applause*

284
00:25:31,810 --> 00:25:34,587
and once again my apologies that
that didn't work

285
00:25:34,587 --> 00:25:52,230
*34c3 outro*