ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo
iOS Continuous Testing
       For fun and pro?t
Ideas, not tools
Ideas, not tools
    Okay, some tools
Why test?
Why test?
1 in X ¡°?xed¡± bugs are not really ?xed
Why test?
1 in 4 ¡°?xed¡± bugs are not really ?xed
Why test?
1 in X ¡°implemented¡± features break the build
Why test?
1 in 5 ¡°implemented¡± features break the build
Why test?
Over ___% of all reopened issues are reopened more
                     than once
Why test?
Over half of all reopened issues are reopened more
                      than once
Why test?
__% of all reopened issues are reopened 3 or more
                       times
Why test?
25% of all reopened issues are reopened 3 or more
                       times
Unit Testing (SenTest)
iOS App Development Work?ow Guide:
         Unit Testing Applications
UI Automation is
    AWFUL!
UI Automation is
         AWFUL!
? I¡¯ve personally ?led over 15 bugs.
UI Automation is
         AWFUL!
? I¡¯ve personally ?led over 15 bugs.
 ? All are still open.
UI Automation is
         AWFUL!
? I¡¯ve personally ?led over 15 bugs.
 ? All are still open.
? 4.2 broke more than it ?xed
UI Automation is
         AWFUL!
? I¡¯ve personally ?led over 15 bugs.
 ? All are still open.
? 4.2 broke more than it ?xed
 ? Continuing the trend of fail
Time spent since August

60

45

30

15

 0
     Instruments Bug Tracker Blogging   HN   Reddit   RSS   Email
Integration Testing (KIF)
iOS Continuous Testing
iOS Continuous Testing
iOS Continuous Testing
iOS Continuous Testing
iOS Continuous Testing
iOS Continuous Testing
Step 1
Step 1

?   NSMutableArray *steps = [NSMutableArray array];
Step 1

  ?   NSMutableArray *steps = [NSMutableArray array];

  [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"Create
Album"]];
Step 1

  ?   NSMutableArray *steps = [NSMutableArray array];

   [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"Create
Album"]];
   [steps addObject:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:@"createAlbumView"]];
Step 1

  ?   NSMutableArray *steps = [NSMutableArray array];

   [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"Create
Album"]];
   [steps addObject:[KIFTestStep
stepToWaitForViewWithAccessibilityLabel:@"createAlbumView"]];

  ?
Step 2
Step 2


[steps addObject:[KIFTestStep stepToEnterText:name
intoViewWithAccessibilityLabel:@"albumName"]];
Step 3
Step 3

if (primary) [steps addObject:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"albumPrimary"]];
Step 3

  if (primary) [steps addObject:[KIFTestStep
  stepToTapViewWithAccessibilityLabel:@"albumPrimary"]];

   [steps addObject:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"createAlbumButton"]];
Step 3

  if (primary) [steps addObject:[KIFTestStep
  stepToTapViewWithAccessibilityLabel:@"albumPrimary"]];

   [steps addObject:[KIFTestStep
stepToTapViewWithAccessibilityLabel:@"createAlbumButton"]];

   [steps addObject:[KIFTestStep
stepToWaitForAbsenceOfViewWithAccessibilityLabel:@"createAlbumVi
ew"]];
Step 4
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

 Global *g = [Global shared];
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

 Global *g = [Global shared];
 BOOL found = NO;
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

 Global *g = [Global shared];
 BOOL found = NO;
 for(Album *a in g.albums){
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

 Global *g = [Global shared];
 BOOL found = NO;
 for(Album *a in g.albums){
    if ([a.name isEqualToString:name]){
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

 Global *g = [Global shared];
 BOOL found = NO;
 for(Album *a in g.albums){
    if ([a.name isEqualToString:name]){
        found = YES;
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

 Global *g = [Global shared];
 BOOL found = NO;
 for(Album *a in g.albums){
    if ([a.name isEqualToString:name]){
        found = YES;
    }
Step 4
[steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

 Global *g = [Global shared];
 BOOL found = NO;
 for(Album *a in g.albums){
    if ([a.name isEqualToString:name]){
        found = YES;
    }
 }
Step 4
    [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
    executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

    Global *g = [Global shared];
    BOOL found = NO;
    for(Album *a in g.albums){
       if ([a.name isEqualToString:name]){
           found = YES;
       }
    }
    KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested
album %@",g.albums]);
Step 4
    [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
    executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

    Global *g = [Global shared];
    BOOL found = NO;
    for(Album *a in g.albums){
       if ([a.name isEqualToString:name]){
           found = YES;
       }
    }
    KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested
album %@",g.albums]);
    return KIFTestStepResultSuccess;
Step 4
    [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
    executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

       Global *g = [Global shared];
       BOOL found = NO;
       for(Album *a in g.albums){
          if ([a.name isEqualToString:name]){
              found = YES;
          }
       }
      KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested
album %@",g.albums]);
       return KIFTestStepResultSuccess;
   }]];
Step 4
    [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
    executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

       Global *g = [Global shared];
       BOOL found = NO;
       for(Album *a in g.albums){
          if ([a.name isEqualToString:name]){
              found = YES;
          }
       }
      KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested
album %@",g.albums]);
       return KIFTestStepResultSuccess;
   }]];
Step 4
    [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation"
    executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {

       Global *g = [Global shared];
       BOOL found = NO;
       for(Album *a in g.albums){
          if ([a.name isEqualToString:name]){
              found = YES;
          }
       }
      KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested
album %@",g.albums]);
       return KIFTestStepResultSuccess;
   }]];


  ?
Step 5
Step 5

#if RUN_KIF_TESTS
Step 5

    #if RUN_KIF_TESTS
   [[PhotoWalletTestController sharedInstance]
startTestingWithCompletionBlock:^{
Step 5

    #if RUN_KIF_TESTS
   [[PhotoWalletTestController sharedInstance]
startTestingWithCompletionBlock:^{
      exit([[PhotoWalletTestController sharedInstance]
failureCount]);
Step 5

    #if RUN_KIF_TESTS
   [[PhotoWalletTestController sharedInstance]
startTestingWithCompletionBlock:^{
       exit([[PhotoWalletTestController sharedInstance]
failureCount]);
   }];
Step 5

    #if RUN_KIF_TESTS
   [[PhotoWalletTestController sharedInstance]
startTestingWithCompletionBlock:^{
       exit([[PhotoWalletTestController sharedInstance]
failureCount]);
   }];
#endif
iOS Continuous Testing
iOS Continuous Testing
iOS Continuous Testing
Keep It Functional



https://github.com/square/KIF
Why continuously test?
Why continuously test?
Why continuously test?
      Don¡¯t break the build!
Why continuously test?
     Don¡¯t hold up deployment!
Why continuously test?
     Refactor with con?dence!
Myth #1: I don¡¯t have time
                Variations



? I¡¯m a one-man shop
? My iOS game is not mission-critical
? Testing takes time away from writing new
  code
Bugs   Tests
Bugs               Tests
70

56

42

28

14

 0
     1.2   1.3     1.4   1.5           1.6   1.7
Continuous Testing
            Bugs                  Tests
70

56

42

28

14

 0
     1.2   1.3       1.4    1.5           1.6   1.7
Continuous Testing
            Bugs                  Tests
70

56

42

28

14

 0
     1.2   1.3       1.4    1.5           1.6   1.7
Ship Dates Over Time


?
Ship Dates Over Time
       Continuous Testing



?
Ship Dates Over Time
          Continuous Testing



?

     Shipping 40 Days Sooner
Experimenting with
      TDD
Experimenting with
      TDD
Experimenting with
      TDD
Experimenting with
      TDD
Experimenting with
      TDD
Experimenting with
      TDD
Nobody likes to test

? We need something fun
? We need something easy
? We need something automatic
? We need something obviously bene?cial
Hudson / Jenkins
Enterprise tools are not your friend
iOS Continuous Testing
iOS Continuous Testing
Enterprise tools are not your friend
Introducing buildbot
      Testing made fun
GLaDOS is alive!
     She sends mail
GLaDOS is alive!
   She uses our bugtracker
GLaDOS is alive!
   She taunts developers
GLaDOS is alive!
  She merges things on GitHub
GLaDOS is alive!
     She ?les bugs
buildbot in practice
      Testing made easy
zero-friction testing
zero-friction testing

? Test by default
zero-friction testing

? Test by default
? Command-line support w/ work.py
zero-friction testing

? Test by default
? Command-line support w/ work.py
? One-click testing from the bug tracker
zero-friction testing

? Test by default
? Command-line support w/ work.py
? One-click testing from the bug tracker
? Human tester selected automagically to
  code review every patch
zero-friction testing
zero-friction testing
? Errors
zero-friction testing
? Errors
? Warnings
zero-friction testing
? Errors
? Warnings
? Analyzer results
zero-friction testing
? Errors
? Warnings
? Analyzer results
? Command-U tests
zero-friction testing
? Errors
? Warnings
? Analyzer results
? Command-U tests
? Merge failures
zero-friction testing
  ? Errors
  ? Warnings
  ? Analyzer results
  ? Command-U tests
  ? Merge failures
  ? KIF UI integration tests
All before a reviewer looks at the patch
zero-friction testing
Uni?ed error summary




  Detailed log ?les
test once, test forever

 ? Reviewers only get sane patches
 ? Once you write a failing test, never
   look at the patch again
Integration made easy
When the tests pass, GLaDOS merges the feature in
              and closes the ticket.
AL
     RN LY
  TE ON
IN E
  USDeployments made easy




                 Only one person knows
                     how to do this!
AL
     RN LY
  TE ON
IN E
  USDeployments made easy
AL
     RN LY
  TE ON
IN E
  USDeployments made easy
AL
     RN LY
  TE ON
IN E
  USDeployments made easy
AL
     RN LY
  TE ON
IN E
  USDeployments made easy
AL
     RN LY
  TE ON
IN E
  USDeployments made easy
AL
     RN LY
  TE ON
IN E
  USDeployments made easy
Setup
buildbot: the bad news
     we¡¯re the only user
buildbot: the bad news

 ? Large buy-in
  ? work.py - work?ow
  ? git - version control
  ? FogBugz - bug tracking
  ? GitHub - source hosting
buildbot: the bad news
  Lots of our defaults aren¡¯t con?gurable
buildbot: the bad news
 Hard to retro?t into your existing work?ow
buildbot: the bad news
    Lots of undocumented behavior
EN E
        OP RC
        S OUbuildbot
http://github.com/drewcrawford/buildbot
OK, now what?
OK, now what?

? Test.
OK, now what?

? Test.
? Test early, test often.
OK, now what?

? Test.
? Test early, test often.
? Test reasonably.
OK, now what?

? Test.
? Test early, test often.
? Test reasonably.
? Test automatically.
OK, now what?

? Test.
? Test early, test often.
? Test reasonably.
? Test automatically.
? Think about your work?ow
?   Tiny iOS Developer

?   Mix of contracts &
    products

?   Many other dev tools
    like buildbot
Drew Crawford
http://drewcrawfordapps.com
drew@drewcrawfordapps.com


http://sealedabstract.com

More Related Content

iOS Continuous Testing

  • 1. iOS Continuous Testing For fun and pro?t
  • 3. Ideas, not tools Okay, some tools
  • 5. Why test? 1 in X ¡°?xed¡± bugs are not really ?xed
  • 6. Why test? 1 in 4 ¡°?xed¡± bugs are not really ?xed
  • 7. Why test? 1 in X ¡°implemented¡± features break the build
  • 8. Why test? 1 in 5 ¡°implemented¡± features break the build
  • 9. Why test? Over ___% of all reopened issues are reopened more than once
  • 10. Why test? Over half of all reopened issues are reopened more than once
  • 11. Why test? __% of all reopened issues are reopened 3 or more times
  • 12. Why test? 25% of all reopened issues are reopened 3 or more times
  • 14. iOS App Development Work?ow Guide: Unit Testing Applications
  • 16. UI Automation is AWFUL! ? I¡¯ve personally ?led over 15 bugs.
  • 17. UI Automation is AWFUL! ? I¡¯ve personally ?led over 15 bugs. ? All are still open.
  • 18. UI Automation is AWFUL! ? I¡¯ve personally ?led over 15 bugs. ? All are still open. ? 4.2 broke more than it ?xed
  • 19. UI Automation is AWFUL! ? I¡¯ve personally ?led over 15 bugs. ? All are still open. ? 4.2 broke more than it ?xed ? Continuing the trend of fail
  • 20. Time spent since August 60 45 30 15 0 Instruments Bug Tracker Blogging HN Reddit RSS Email
  • 29. Step 1 ? NSMutableArray *steps = [NSMutableArray array];
  • 30. Step 1 ? NSMutableArray *steps = [NSMutableArray array]; [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"Create Album"]];
  • 31. Step 1 ? NSMutableArray *steps = [NSMutableArray array]; [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"Create Album"]]; [steps addObject:[KIFTestStep stepToWaitForViewWithAccessibilityLabel:@"createAlbumView"]];
  • 32. Step 1 ? NSMutableArray *steps = [NSMutableArray array]; [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"Create Album"]]; [steps addObject:[KIFTestStep stepToWaitForViewWithAccessibilityLabel:@"createAlbumView"]]; ?
  • 34. Step 2 [steps addObject:[KIFTestStep stepToEnterText:name intoViewWithAccessibilityLabel:@"albumName"]];
  • 36. Step 3 if (primary) [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"albumPrimary"]];
  • 37. Step 3 if (primary) [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"albumPrimary"]]; [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"createAlbumButton"]];
  • 38. Step 3 if (primary) [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"albumPrimary"]]; [steps addObject:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"createAlbumButton"]]; [steps addObject:[KIFTestStep stepToWaitForAbsenceOfViewWithAccessibilityLabel:@"createAlbumVi ew"]];
  • 40. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {
  • 41. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) {
  • 42. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared];
  • 43. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO;
  • 44. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){
  • 45. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){
  • 46. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){ found = YES;
  • 47. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){ found = YES; }
  • 48. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){ found = YES; } }
  • 49. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){ found = YES; } } KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested album %@",g.albums]);
  • 50. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){ found = YES; } } KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested album %@",g.albums]); return KIFTestStepResultSuccess;
  • 51. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){ found = YES; } } KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested album %@",g.albums]); return KIFTestStepResultSuccess; }]];
  • 52. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){ found = YES; } } KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested album %@",g.albums]); return KIFTestStepResultSuccess; }]];
  • 53. Step 4 [steps addObject:[KIFTestStep stepWithDescription:@"Testing album creation" executionBlock:^KIFTestStepResult(KIFTestStep *step, NSError *__autoreleasing *error) { Global *g = [Global shared]; BOOL found = NO; for(Album *a in g.albums){ if ([a.name isEqualToString:name]){ found = YES; } } KIFTestCondition(found, error, [NSString stringWithFormat:@"Failed to create requested album %@",g.albums]); return KIFTestStepResultSuccess; }]]; ?
  • 56. Step 5 #if RUN_KIF_TESTS [[PhotoWalletTestController sharedInstance] startTestingWithCompletionBlock:^{
  • 57. Step 5 #if RUN_KIF_TESTS [[PhotoWalletTestController sharedInstance] startTestingWithCompletionBlock:^{ exit([[PhotoWalletTestController sharedInstance] failureCount]);
  • 58. Step 5 #if RUN_KIF_TESTS [[PhotoWalletTestController sharedInstance] startTestingWithCompletionBlock:^{ exit([[PhotoWalletTestController sharedInstance] failureCount]); }];
  • 59. Step 5 #if RUN_KIF_TESTS [[PhotoWalletTestController sharedInstance] startTestingWithCompletionBlock:^{ exit([[PhotoWalletTestController sharedInstance] failureCount]); }]; #endif
  • 66. Why continuously test? Don¡¯t break the build!
  • 67. Why continuously test? Don¡¯t hold up deployment!
  • 68. Why continuously test? Refactor with con?dence!
  • 69. Myth #1: I don¡¯t have time Variations ? I¡¯m a one-man shop ? My iOS game is not mission-critical ? Testing takes time away from writing new code
  • 70. Bugs Tests
  • 71. Bugs Tests 70 56 42 28 14 0 1.2 1.3 1.4 1.5 1.6 1.7
  • 72. Continuous Testing Bugs Tests 70 56 42 28 14 0 1.2 1.3 1.4 1.5 1.6 1.7
  • 73. Continuous Testing Bugs Tests 70 56 42 28 14 0 1.2 1.3 1.4 1.5 1.6 1.7
  • 74. Ship Dates Over Time ?
  • 75. Ship Dates Over Time Continuous Testing ?
  • 76. Ship Dates Over Time Continuous Testing ? Shipping 40 Days Sooner
  • 83. Nobody likes to test ? We need something fun ? We need something easy ? We need something automatic ? We need something obviously bene?cial
  • 84. Hudson / Jenkins Enterprise tools are not your friend
  • 87. Enterprise tools are not your friend
  • 88. Introducing buildbot Testing made fun
  • 89. GLaDOS is alive! She sends mail
  • 90. GLaDOS is alive! She uses our bugtracker
  • 91. GLaDOS is alive! She taunts developers
  • 92. GLaDOS is alive! She merges things on GitHub
  • 93. GLaDOS is alive! She ?les bugs
  • 94. buildbot in practice Testing made easy
  • 97. zero-friction testing ? Test by default ? Command-line support w/ work.py
  • 98. zero-friction testing ? Test by default ? Command-line support w/ work.py ? One-click testing from the bug tracker
  • 99. zero-friction testing ? Test by default ? Command-line support w/ work.py ? One-click testing from the bug tracker ? Human tester selected automagically to code review every patch
  • 103. zero-friction testing ? Errors ? Warnings ? Analyzer results
  • 104. zero-friction testing ? Errors ? Warnings ? Analyzer results ? Command-U tests
  • 105. zero-friction testing ? Errors ? Warnings ? Analyzer results ? Command-U tests ? Merge failures
  • 106. zero-friction testing ? Errors ? Warnings ? Analyzer results ? Command-U tests ? Merge failures ? KIF UI integration tests All before a reviewer looks at the patch
  • 107. zero-friction testing Uni?ed error summary Detailed log ?les
  • 108. test once, test forever ? Reviewers only get sane patches ? Once you write a failing test, never look at the patch again
  • 109. Integration made easy When the tests pass, GLaDOS merges the feature in and closes the ticket.
  • 110. AL RN LY TE ON IN E USDeployments made easy Only one person knows how to do this!
  • 111. AL RN LY TE ON IN E USDeployments made easy
  • 112. AL RN LY TE ON IN E USDeployments made easy
  • 113. AL RN LY TE ON IN E USDeployments made easy
  • 114. AL RN LY TE ON IN E USDeployments made easy
  • 115. AL RN LY TE ON IN E USDeployments made easy
  • 116. AL RN LY TE ON IN E USDeployments made easy
  • 117. Setup
  • 118. buildbot: the bad news we¡¯re the only user
  • 119. buildbot: the bad news ? Large buy-in ? work.py - work?ow ? git - version control ? FogBugz - bug tracking ? GitHub - source hosting
  • 120. buildbot: the bad news Lots of our defaults aren¡¯t con?gurable
  • 121. buildbot: the bad news Hard to retro?t into your existing work?ow
  • 122. buildbot: the bad news Lots of undocumented behavior
  • 123. EN E OP RC S OUbuildbot http://github.com/drewcrawford/buildbot
  • 125. OK, now what? ? Test.
  • 126. OK, now what? ? Test. ? Test early, test often.
  • 127. OK, now what? ? Test. ? Test early, test often. ? Test reasonably.
  • 128. OK, now what? ? Test. ? Test early, test often. ? Test reasonably. ? Test automatically.
  • 129. OK, now what? ? Test. ? Test early, test often. ? Test reasonably. ? Test automatically. ? Think about your work?ow
  • 130. ? Tiny iOS Developer ? Mix of contracts & products ? Many other dev tools like buildbot

Editor's Notes