Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(91)

Side by Side Diff: 2012/splash.slide

Issue 6777060: code review 6777060: go.talks: add 'go at google' (Closed)
Patch Set: Created 11 years, 5 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | 2012/splash/appenginegophercolor.jpg » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
(Empty)
1 Go at Google
2 SPLASH, Tucson, Oct 25, 2012
3
4 Rob Pike
5 Google, Inc.
6 @rob_pike
7 http://golang.org/s/plusrob
8 http://golang.org
9
10 * Preamble
11
12 * What is Go?
13
14 Go is:
15
16 - open source
17 - concurrent
18 - garbage-collected
19 - efficient
20 - scalable
21 - simple
22 - fun
23 - boring (to some)
24
25 .link http://golang.org http://golang.org
26
27 * History
28
29 Design began in late 2007.
30
31 Key players:
32
33 - Robert Griesemer, Rob Pike, Ken Thompson
34 - Later: Ian Lance Taylor, Russ Cox
35
36 Became open source in November 2009.
37
38 Developed entirely in the open; very active community.
39 Language stable as of Go 1, early 2012.
40
41 * Go at Google
42
43 Go is a programming language designed by Google to help solve Google's problems.
44
45 Google has big problems.
46
47 * Big hardware
48
49 .image splash/datacenter.jpg
50
51 * Big software
52
53 - C++ (mostly) for servers, plus lots of Java and Python
54 - thousands of engineers
55 - gazillions of lines of code
56 - distributed build system
57 - one tree
58
59 And of course:
60
61 - zillions of machines, which we treat as a modest number of compute clusters
62
63 Development at Google can be slow, often clumsy.
64
65 But it _is_ effective.
66
67 * The reason for Go
68
69 Goals:
70
71 - eliminate slowness
72 - eliminate clumsiness
73 - improve effectiveness
74 - maintain (even improve) scale
75
76 Go was designed by and for people who write—and read and debug and maintain—larg e software systems.
77
78 Go's purpose is _not_ research into programming language design.
79
80 Go's purpose is to make its designers' programming lives better.
81
82 * Today's theme
83
84 A talk about software engineering more than language design.
85
86 To be more accurate:
87
88 - Language design in the service of software engineering.
89
90 In short:
91
92 - How does a language help software engineering?
93
94 * Features?
95
96 Reaction upon launch:
97 My favorite feature isn't in Go! Go Sucks!
98
99 This misses the point.
100
101 * Pain
102
103 What makes large-scale development hard with C++ or Java (at least):
104
105 - slow builds
106 - uncontrolled dependencies
107 - each programmer using a different subset of the language
108 - poor program understanding (documentation, etc.)
109 - duplication of effort
110 - cost of updates
111 - version skew
112 - difficulty of automation (auto rewriters etc.): tooling
113 - cross-language builds
114
115 Language _features_ don't usually address these.
116
117 * Focus
118
119 In the design of Go, we tried to focus on solutions to _these_ problems.
120
121 Example: indentation for structure vs. C-like braces
122
123 * Dependencies in C and C++
124
125 * A personal history of dependencies in C
126
127 `#ifdef`
128
129 `#ifndef` "guards":
130
131 #ifndef _SYS_STAT_H_
132
133 1984: `<sys/stat.h>` times 37
134
135 ANSI C and `#ifndef` guards:
136
137 - dependencies accumulate
138 - throw includes at the program until it compiles
139 - no way to know what can be removed
140
141 * A personal history of dependencies in Plan 9's C
142
143 Plan 9, another take:
144
145 - no `#ifdefs` (or `#ifndefs`)
146 - documentation and topological sort
147 - easy to find out what can be removed
148
149 Need to document dependencies, but much faster compilation.
150
151 In short:
152
153 - _ANSI_C_made_a_costly_mistake_ in requiring `#ifndef` guards.
154
155 * A personal history of dependencies in C++
156
157 C++ exacerbated the problem:
158
159 - one `#include` file per class
160 - code (not just declarations) in `#include` files
161 - `#ifndef` guards persist
162
163 2004: Mike Burrows and Chubby: `<xxx>` times 37,000
164
165 1984: Tom Cargill and pi
166
167 * A personal history of dependencies at Google
168
169 Plan 9 demo: a story
170
171 Early Google: one `Makefile`
172
173 2003: `Makefile` generated from per-directory `BUILD` files
174
175 - explicit dependencies
176 - 40% smaller binaries
177
178 Dependencies still not checkable!
179
180 * Result
181
182 To build a large Google binary on a single computer is impractical.
183
184 In 2007, instrumented the build of a major Google binary:
185
186 - 2000 files
187 - 4.2 megabytes·
188 - 8 gigabytes delivered to compiler
189 - 2000 bytes sent to compiler for every C++ source byte
190 - it's real work too: `<string>` for example
191 - hours to build
192
193 * Tools can help
194
195 New distributed build system:
196
197 - no more `Makefile` (still uses `BUILD` files)
198 - many buildbots
199 - much caching
200 - much complexity (a large program in its own right)
201
202 Even with Google's massive distributed build system, a large build still takes m inutes.
203 (In 2007 that binary took 45 minutes; today, 27 minutes.)
204
205 Poor quality of life.
206
207 * Enter Go
208
209 While that build runs, we have time to think.
210
211 Want a language to improve the quality of life.
212
213 And dependencies are only one such problem....
214
215 * Primary considerations
216
217 Must work at scale:
218
219 - large programs
220 - large teams
221 - large number of dependencies
222
223 Must be familiar, roughly C-like
224
225 * Modernize
226
227 The old ways are _old_.
228
229 Go should be:
230
231 - suitable for multicore machines
232 - suitable for networked machines
233 - suitable for web stuff
234
235 * The design of Go
236
237 From a software engineering perspective.
238
239 * Dependencies in Go
240
241 * Dependencies
242
243 Dependencies are defined (syntactically) in the language.
244
245 Explicit, clear, computable.
246
247 import "encoding/json"
248
249 Unused dependencies cause error at compile time.
250
251 Efficient: dependencies traversed once per source file...
252
253 * Hoisting dependencies
254
255 Consider:
256 `A` imports `B` imports `C` but `A` does not directly import `C`.
257
258 The object code for `B` includes all the information about `C` needed to import `B`.
259 Therefore in `A` the line
260
261 import "B"
262
263 does not require the compiler to read `C` when compiling `A`.
264
265 Also, the object files are designed so the "export" information comes first; com piler doing import does not need to read whole file.
266
267 Exponentially less data read than with `#include` files.
268
269 With Go in Google, about 40X fanout (recall C++ was 2000x)
270 Plus in C++ it's general code that must be parsed; in Go it's just export data.
271
272 * No circular imports
273
274 Circular imports are illegal in Go.
275
276 The big picture in a nutshell:
277
278 - occasional minor pain,
279 - but great reduction in annoyance overall·
280 - structural typing makes it less important than with type hierarchies
281 - keeps us honest!
282
283 Forces clear demarcation between packages.
284
285 Simplifies compilation, linking, initialization.
286
287 * API design
288
289 Through the design of the standard library, great effort spent on controlling de pendencies.
290
291 It can be better to copy a little code than to pull in a big library for one fun ction.
292 (A test in the system build complains if new core dependencies arise.)
293
294 Dependency hygiene trumps code reuse.
295
296 Example:
297 The (low-level) `net` package has own `itoa` to avoid dependency on the big form atted I/O package.
298
299 * Packages
300
301 * Packages
302
303 Every Go source file, e.g. `"encoding/json/json.go"`, starts
304
305 package json
306
307 where `json` is the "package name", an identifier.
308 Package names are usually concise.
309
310 To use a package, need to identify it by path:
311
312 import "encoding/json"
313
314 And then the package name is used to qualify items from package:
315
316 var dec = json.NewDecoder(reader)
317
318 Clarity: can always tell if name is local to package from its syntax: `Name` vs. `pkg.Name`.
319 (More on this later.)
320
321 Package combines properties of library, name space, and module.
322
323 * Package paths are unique, not package names
324
325 The path is `"encoding/json"` but the package name is `json`.
326 The path identifies the package and must be unique.
327 Project or company name at root of name space.
328
329 import "google/base/go/log"
330
331 Package name might not be unique; can be overridden. These are both `package`log `:
332
333 import "log" // Standard package
334 import googlelog "google/base/go/log" // Google-specific package
335
336 Every company might have its own `log` package; no need to make the package name unique.
337
338 Another: there are many `server` packages in Google's code base.
339
340 * Remote packages
341
342 Package path syntax works with remote repositories.
343 The import path is just a string.
344
345 Can be a file, can be a URL:
346
347 go get github.com/4ad/doozer // Command to fetch package
348
349 import "github.com/4ad/doozer" // Doozer client's import statement
350
351 var client doozer.Conn // Client's use of package
352
353 * Go's Syntax
354
355 * Syntax
356
357 Syntax is not important...
358
359 - unless you're programming
360 - or writing tools
361
362 Tooling is essential, so Go has a clean syntax.
363 Not super small, just clean:
364
365 - regular (mostly)
366 - only 25 keywords
367 - straightforward to parse (no type-specific context required)
368 - easy to predict, reason about
369
370 * Declarations
371
372 Uses Pascal/Modula-style syntax: name before type, more type keywords.
373
374 var fn func([]int) int
375 type T struct { a, b int }
376
377 not
378
379 int (*fn)(int[]);
380 struct T { int a, b; }
381
382 Easier to parse—no symbol table needed. Tools become easier to write.
383
384 One nice effect: can drop `var` and derive type of variable from expression:
385
386 var buf *bytes.Buffer = bytes.NewBuffer(x) // explicit
387 buf := bytes.NewBuffer(x) // derived
388
389 For more information:
390
391 .link http://golang.org/s/decl-syntax
392
393 * Function syntax
394
395 Function on type `T`:
396
397 func Abs(t T) float64
398
399 Method of type `T`:
400
401 func (t T) Abs() float64
402
403 Variable (closure) of type `T`:
404
405 negAbs := func(t T) float64 { return -Abs(t) }
406 ········
407 In Go, functions can return multiple values. Common case: `error`.
408
409 func ReadByte() (c byte, err error)
410
411 c, err := ReadByte()
412 if err != nil { ... }
413
414 More about errors later.
415
416 * No default arguments
417
418 Go does not support default function arguments.
419
420 Why not?
421
422 - too easy to throw in defaulted args to fix design problems
423 - encourages too many args
424 - too hard to understand the effect of the fn for different combinations of args
425
426 Extra verbosity may happen but that encourages extra thought about names.
427
428 Related: Go has easy-to-use, type-safe support for variadic functions.
429
430 * Naming
431
432 * Export syntax
433
434 Simple rule:
435
436 - upper case initial letter: `Name` is visible to clients of package
437 - otherwise: `name` (or `_Name`) is not visible to clients of package
438
439 Applies to variables, types, functions, methods, constants, fields....
440
441 That Is It.
442
443 Not an easy decision.
444 One of the most important things about the language.
445
446 Can see the visibility of an identifier without discovering the declaration.
447
448 Clarity.
449
450 * Scope
451
452 Go has very simple scope hierarchy:
453
454 - universe
455 - package
456 - file (for imports only)
457 - function
458 - block
459
460 * Locality of naming
461
462 Nuances:
463
464 - no implicit `this` in methods (receiver is explicit); always see `rcvr.Field`
465 - package qualifier always present for imported names
466 - (first component of) every name is always declared in current package
467
468 No surprises when importing:
469
470 - adding an exported name to my package cannot break your package!
471
472 Names do not leak across boundaries.
473
474 In C, C++, Java the name `y` could refer to anything
475 In Go, `y` (or even `Y`) is always defined within the package.
476 In Go, `x.Y` is clear: find `x` locally, `Y` belongs to it.
477
478 * Function and method lookup
479
480 Method lookup by name only, not type.
481 A type cannot have two methods with the same name, ever.
482 Easy to identify which function/method is referred to.
483 Simple implementation, simpler program, fewer surprises.
484
485 Given a method `x.M`, there's only ever one `M` associated with `x`.
486
487 * Semantics
488
489 * Basics
490
491 Generally C-like:
492
493 - statically typed
494 - procedural
495 - compiled
496 - pointers etc.
497
498 Should feel familiar to programmers from the C family.
499
500 * But...
501
502 Many small changes in the aid of robustness:
503
504 - no pointer arithmetic
505 - no implicit numeric conversions
506 - array bounds checking
507 - no type aliases
508 - `++` and `--` as statements not expressions
509 - assignment not an expression
510 - legal (encouraged even) to take address of stack variable
511 - many more
512
513 Plus some big ones...
514
515 * Bigger things
516
517 Some elements of Go step farther from C, even C++ and Java:
518
519 - concurrency
520 - garbage collection
521 - interface types
522 - reflection
523 - type switches
524
525 * Concurrency
526
527 * Concurrency
528
529 Important to modern computing environment.
530 Not well served by C++ or even Java.
531
532 Go embodies a variant of CSP with first-class channels.
533
534 Why CSP?
535
536 - The rest of the language can be ordinary and familiar.
537
538 Must be able to couple concurrency with computation.
539
540 Example: concurrency and cryptography.
541
542 * CSP is practical
543
544 For a web server, the canonical Go program, the model is a great fit.
545
546 Go _enables_ simple, safe concurrent programming.
547 It doesn't _forbid_ bad programming.
548
549 Focus on _composition_ of regular code.
550
551 Caveat: not purely memory safe; sharing is legal.
552 Passing a pointer over a channel is idiomatic.
553
554 Experience shows this is a practical design.
555
556 * Garbage collection
557
558 * The need for garbage collection
559
560 Too much programming in C and C++ is about memory allocation.
561 But also the design revolves too much about memory management.
562 Leaky abstractions, leaky dependencies.
563
564 Go has garbage collection, only.
565
566 Needed for concurrency: tracking ownership too hard otherwise.
567 Important for abstraction: separate behavior from resource management.
568 A key part of scalability: APIs remain local.
569
570 Use of the language is much simpler because of GC.
571 Adds run-time cost, latency, complexity to the implementation.
572
573 Day 1 design decision.
574
575 * Garbage collection in Go
576
577 A garbage-collected systems language is heresy!
578 Experience with Java: Uncontrollable cost, too much tuning.
579
580 But Go is different.
581 Go lets you limit allocation by controlling memory layout.
582
583 Example:
584
585 type X struct {
586 a, b, c int
587 buf [256]byte
588 }
589
590 Example: Custom arena allocator with free list.
591
592 * Interior pointers
593
594 Early decision: allow interior pointers such as `X.buf` from previous slide.
595
596 Tradeoff: Affects which GC algorithms that can be used, but in return reduces pr essure on the collector.
597
598 Gives the _programmer_ tools to control GC overhead.
599
600 Experience, compared to Java, shows it has significant effect on memory pressure .
601
602 GC remains an active subject.
603 Current design: parallel mark-and-sweep.
604 With care to use memory wisely, works well in production.
605
606 * Interfaces
607
608 Composition not inheritance
609
610 * Object orientation and big software
611
612 Go is object-oriented.
613 Go does not have classes or subtype inheritance.
614
615 What does this mean?
616
617 * No type hierarchy
618
619 O-O is important because it provides uniformity of interface.
620 Outrageous example: the Plan 9 kernel.
621
622 Problem: subtype inheritance encourages _non-uniform_ interfaces.
623
624 * O-O and program evolution
625
626 Design by type inheritance oversold.
627 Generates brittle code.
628 Early decisions hard to change, often poorly informed.
629 Makes every programmer an interface designer.
630 (Plan 9 was built around a single interface everything needed to satisfy.)
631
632 Therefore encourages overdesign early on: predict every eventuality.
633 Exacerbates the problem, complicates designs.
634
635 * Go: interface composition
636
637 In Go an interface is _just_ a set of methods:
638
639 type Hash interface {
640 Write(p []byte) (n int, err error)
641 Sum(b []byte) []byte
642 Reset()
643 Size() int
644 BlockSize() int
645 }
646
647 No `implements` declaration.
648 All hash implementations satisfy this implicitly. (Statically checked.)
649
650 * Interfaces in practice: composition
651
652 Tend to be small: one or two methods are common.
653
654 Composition falls out trivially. Easy example, from package `io`:
655
656 type Reader interface {
657 Read(p []byte) (n int, err error)
658 }
659
660 `Reader` (plus the complementary `Writer`) makes it easy to chain:
661
662 - files, buffers, networks, encryptors, compressors, GIF, JPEG, PNG, ...
663
664 Dependency structure is not a hierarchy; these also implement other interfaces.
665
666 Growth through composition is _natural_, does not need to be pre-declared.
667
668 And that growth can be _ad_hoc_ and linear.
669
670 * Compose with functions, not methods
671
672 Hard to overstate the effect that Go's interfaces have on program design.
673
674 One big effect: functions with interface arguments.
675
676 func ReadAll(r io.Reader) ([]byte, error)
677
678 Wrappers:
679
680 func LoggingReader(r io.Reader) io.Reader
681 func LimitingReader(r io.Reader, n int64) io.Reader
682 func ErrorInjector(r io.Reader) io.Reader
683
684 The designs are nothing like hierarchical, subtype-inherited methods.
685 Much looser, organic, decoupled, independent.
686
687 * Errors
688
689 * Error handling
690
691 Multiple function return values inform the design for handling errors.
692
693 Go has no `try-catch` control structures for exceptions.
694 Return `error` instead: built-in interface type that can "stringify" itself:
695
696 type error interface { Error() string }
697
698 Clear and simple.
699
700 Philosophy:
701
702 Forces you think about errors—and deal with them—when they arise.
703 Errors are _normal_. Errors are _not_exceptional_.
704 Use the existing language to compute based on them.
705 Don't need a sublanguage that treats them as exceptional.
706
707 Result is better code (if more verbose).
708
709 * (OK, not all errors are normal. But most are.)
710
711 .image splash/fire.jpg
712
713 * Tools
714
715 * Tools
716
717 Software engineering requires tools.
718
719 Go's syntax, package design, naming, etc. make tools easy to write.
720
721 Standard library includes lexer and parser; type checker nearly done.
722
723 * Gofmt
724
725 Always intended to do automatic code formatting.
726 Eliminates an entire class of argument.
727 Runs as a "presubmit" to the code repositories.
728
729 Training:
730
731 - The community has always seen `gofmt` output.
732
733 Sharing:
734
735 - Uniformity of presentation simplifies sharing.
736
737 Scaling:
738
739 - Less time spent on formatting, more on content.
740
741 Often cited as one of Go's best features.
742
743 * Gofmt and other tools
744
745 Surprise: The existence of `gofmt` enabled _semantic_ tools:
746 Can rewrite the tree; `gofix` will clean up output.
747
748 Examples:
749
750 - `gofmt`-r`'a[b:len(a)]`->`a[b:]'`
751 - `gofix`
752
753 And good front-end libraries enable ancillary tools:
754
755 - `godoc`
756 - `go`get`, `go`build`, etc.
757 - `api`
758
759 * Gofix
760
761 The `gofix` tool allowed us to make sweeping changes to APIs and language featur es leading up to the release of Go 1.
762
763 - change to map deletion syntax
764 - new time API
765 - many more
766
767 Also allows us to _update_ code even if the old code still works.
768
769 Recent example:
770
771 Changed Go's protocol buffer implementation to use getter functions; updated _al l_ Google Go code to use them with `gofix`.
772
773 * Conclusion
774
775 * Go at Google
776
777 Go's use is growing inside Google.
778
779 Several big services uses it:
dchest 2012/10/26 20:15:29 s/uses/use/
780
781 - golang.org
782 - youtube.com
783 - dl.google.com
784
785 Many small ones do, many using Google App Engine.
786
787 * Go outside Google
788
789 Many outside companies use it, including:
790
791 - BBC Worldwide
792 - Canonical
793 - Heroku
794 - Nokia
795 - SoundCloud
796
797 * What's wrong?
798
799 Not enough experience yet to know if Go is truly successful.
800 Not enough big programs.
801
802 Some minor details wrong. Examples:
803
804 - declarations still too fussy
805 - `nil` is overloaded
806 - lots of library details
807
808 `Gofix` and `gofmt` gave us the opportunity to fix many problems, ranging from e liminating semicolons to redesigning the `time` package.
809 But we're still learning (and the language is frozen for now).
810
811 The implementation still needs work, the run-time system in particular.
812
813 But all indicators are positive.
814
815 * Summary
816
817 Software engineering guided the design.
818 But a productive, fun language resulted because that design enabled productivity .
819
820 Clear dependencies
821 Clear syntax
822 Clear semantics
823 Composition not inheritance
824 Simplicity of model (GC, concurrency)
825 Easy tooling (the `go` tool, `gofmt`, `godoc`, `gofix`)
826
827 * Try it!
828
829 .link http://golang.org http://golang.org
830
831 .image splash/appenginegophercolor.jpg
832
OLDNEW
« no previous file with comments | « no previous file | 2012/splash/appenginegophercolor.jpg » ('j') | no next file with comments »

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld f62528b