Mopcon2017 - AppDevKit x CameraKit
0 likes485 views
The document outlines a presentation on enhancing camera app development using CameraKit, showcasing its features and performance capabilities. It includes discussions on real-time computing, face detection, customization options, and performance optimization methods for camera applications. The content emphasizes creating an engaging user experience with functionalities such as instant effects and selfie modes.
![15
Initializing a camera
Making Camera Amazing
camera = [[ADKCamera alloc] initCameraWithDelegate:self
quality:AVCaptureSessionPresetLow
position:ADKCameraPositionRear];
camera.alignDeviceOrientation = YES;](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-15-320.jpg)
![17
Camera Viewfinder Preview
Making Camera Amazing
// Adding preview layer on the screen.
[view.layer addSublayer:camera.captureVideoPreviewLayer]
// Starting to capture images.
[camera startCamera];](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-17-320.jpg)
![25Making Camera Amazing
camera = [[ADKCamera alloc] initCameraWithDelegate:self
quality:AVCaptureSessionPresetHigh
position:ADKCameraPositionFront];
camera.trackLiveVideoData = YES;
camera.liveVideoDataDlegate = self;
Getting Real-time Images](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-25-320.jpg)
![27
Doing Face Detection
Making Camera Amazing
- (void)ADKCamera:(ADKCamera *)camera didUpdateSampleBuffer:
(CMSampleBufferRef)sampleBuffer
{
// Converting sample buffer to core image.
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CIImage *sourceCoreImage = [CIImage imageWithCVPixelBuffer:
(CVPixelBufferRef)imageBuffer options:nil];](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-27-320.jpg)
![29
Doing Face Detection
Making Camera Amazing
faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace
context:nil
options:opts];
// Fetching features
NSArray *features = [faceDetector featuresInImage:sourceCoreImage];](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-29-320.jpg)
![31
Using CoreImage
Making Camera Amazing
CIFilter *filter = [CIFilter filterWithName:@CIBumpDistortion"];
[filter setValue:filteredCoreImage forKey:kCIInputImageKey];
[filter setValue:[CIVector vectorWithCGPoint:leftEyePoint]
forKey:kCIInputCenterKey];
[filter setValue:@(eyeRadius) forKey:kCIInputRadiusKey];
[filter setValue:@(0.55) forKey:kCIInputScaleKey];
filteredCoreImage = filter.outputImage;](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-31-320.jpg)
![33
Add Instant Effect
Making Camera Amazing
CIFilter *filter = [CIFilter filterWithName:@"CIPhotoEffectInstant"];
[filter setValue:filteredCoreImage
forKey:kCIInputImageKey];
filteredCoreImage = filter.outputImage;](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-33-320.jpg)
![35
Display Images
Making Camera Amazing
dispatch_async(dispatch_get_main_queue(), ^{
// Actually, this is a bad idea!
UIImage *image = [UIImage imageWithCIImage:filteredCoreImage];
imageView.image = image;
});](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-35-320.jpg)
![45
Using ADKOpenGlImageView
Making Camera Amazing
// Set up config and assign a image for displaying.
glImageView.contentMode =
ADKOpenGLImageViewContentModeScaleAspectFill;
glImageView.backgroundColor = [UIColor black];
glImageView.image = finalCoreImage;](https://image.slidesharecdn.com/mopcon2017-share-171101020000/85/Mopcon2017-AppDevKit-x-CameraKit-45-320.jpg)
Mopcon2017 - AppDevKit x CameraKit
- 1. 1 AppDevKit x CameraKit Anistar Sung - Yahoo Senior Engineer Manager 7 Making Camera Amazing
- 2. Self Introduction 2 Working for Yahoo Top 10 Camera apps - The New York Times AppDevKit iOS open source project iOS app development experience Making Camera Amazing
- 3. What the CameraKit can do? High performance real-time computing with camera CPU & GPU rendering solutions Demos with sample apps 3 Agenda Making Camera Amazing
- 7. Customization Functional limitation Memory usage 7 Issues of Using UIImagePickController API Making Camera Amazing
- 9. What the CameraKit can do? 9Making Camera Amazing
- 10. 10 R-Finder 2013
- 11. 11Making Camera Amazing SNAP & BUY 2014
- 12. 12 CameraKit
- 13. 13 AppDevKit x CameraKit Making Camera Amazing UI Image Camera Animation List Common
- 14. 14Making Camera Amazing ADKCameraPosition ADKCameraFlashMode ADKCameraTorchMode ADKCameraMirrorMode ADKCameraFocusMode ADKCameraExposureMode ADKCameraWhiteBlanceMode ADKCameraErrorCodealignDeviceOrientation trackLiveVideoData cameraQuality captureVideoPreviewLayer recording exposureBias shutterSpeed ISO zoomFactor lensPosition whiteBalanceTemperature whiteBalanceTint lowLightBoost stabilization cameraPermission microphonePermission frontCameraAvailable rearCameraAvailable optimizeForHighestFrameRate flashAvailable torchAvailable focusAtPoint exposureAtPoint startCamera stopCamera captureImage startCaptureVideo stopCaptureVideo ADKCamera
- 15. 15 Initializing a camera Making Camera Amazing camera = [[ADKCamera alloc] initCameraWithDelegate:self quality:AVCaptureSessionPresetLow position:ADKCameraPositionRear]; camera.alignDeviceOrientation = YES;
- 17. 17 Camera Viewfinder Preview Making Camera Amazing // Adding preview layer on the screen. [view.layer addSublayer:camera.captureVideoPreviewLayer] // Starting to capture images. [camera startCamera];
- 18. 18 Taking photos / videos Making Camera Amazing // Capturing a high quality photo. [camera captureImage:^(UIImage *image, NSDictionary *exifDic, NSError *error) { // Using the image to do something }
- 19. 19 Demo Making Camera Amazing Powering up a simple camera app by CameraKit Catherine Shu, Lead Engineer
- 20. High performance real-time computing with camera 20Making Camera Amazing
- 23. 23 Goal Making Camera Amazing 1. Making an awesome effect to make girls happy 2. Could be previewed effects immediately 3. Supporting selfies mode
- 24. 24 Demo Making Camera Amazing Using CameraKit to create a Selfies CAM Catherine Shu, Lead Engineer
- 25. 25Making Camera Amazing camera = [[ADKCamera alloc] initCameraWithDelegate:self quality:AVCaptureSessionPresetHigh position:ADKCameraPositionFront]; camera.trackLiveVideoData = YES; camera.liveVideoDataDlegate = self; Getting Real-time Images
- 27. 27 Doing Face Detection Making Camera Amazing - (void)ADKCamera:(ADKCamera *)camera didUpdateSampleBuffer: (CMSampleBufferRef)sampleBuffer { // Converting sample buffer to core image. CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); CIImage *sourceCoreImage = [CIImage imageWithCVPixelBuffer: (CVPixelBufferRef)imageBuffer options:nil];
- 28. 28 CoreImage
- 29. 29 Doing Face Detection Making Camera Amazing faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:opts]; // Fetching features NSArray *features = [faceDetector featuresInImage:sourceCoreImage];
- 31. 31 Using CoreImage Making Camera Amazing CIFilter *filter = [CIFilter filterWithName:@CIBumpDistortion"]; [filter setValue:filteredCoreImage forKey:kCIInputImageKey]; [filter setValue:[CIVector vectorWithCGPoint:leftEyePoint] forKey:kCIInputCenterKey]; [filter setValue:@(eyeRadius) forKey:kCIInputRadiusKey]; [filter setValue:@(0.55) forKey:kCIInputScaleKey]; filteredCoreImage = filter.outputImage;
- 33. 33 Add Instant Effect Making Camera Amazing CIFilter *filter = [CIFilter filterWithName:@"CIPhotoEffectInstant"]; [filter setValue:filteredCoreImage forKey:kCIInputImageKey]; filteredCoreImage = filter.outputImage;
- 35. 35 Display Images Making Camera Amazing dispatch_async(dispatch_get_main_queue(), ^{ // Actually, this is a bad idea! UIImage *image = [UIImage imageWithCIImage:filteredCoreImage]; imageView.image = image; });
- 36. CPU & GPU rendering solutions 36Making Camera Amazing
- 37. 37 Poor Performance Flow Making Camera Amazing CVPixelBuffer CIImage CVPixelBuffer CIContext EAGLContext UIImage UIImageView
- 38. 38 Requirements in Real-time Process Making Camera Amazing 60 fps Frame 1 Frame 2 Frame 3 Frame 60Frame 4 16.6 ms 16.6 ms 16.6 ms 16.6 ms 16.6 ms
- 41. 41 Better Performance Flow Making Camera Amazing CVPixelBuffer CIImage CVPixelBuffer CIContext EAGLContext GLKView UIImage UIImageView
- 42. 42 ADKOpenGLImageView Making Camera Amazing CIImage *image UIColor *backgroundColor ADKOpenGLImageViewContentMode *contentMode
- 44. 44 Better Performance Flow Making Camera Amazing CVPixelBuffer CIImage CVPixelBuffer CIContext EAGLContext ADKOpenGLImageView UIImage UIImageView
- 45. 45 Using ADKOpenGlImageView Making Camera Amazing // Set up config and assign a image for displaying. glImageView.contentMode = ADKOpenGLImageViewContentModeScaleAspectFill; glImageView.backgroundColor = [UIColor black]; glImageView.image = finalCoreImage;
- 46. 46 Perfomance comparison Making Camera Amazing 300% + * It only for testing the performance of image assignment and rendering
- 47. ADKCamera makes things simple Using GPU to process image will be better (EX: CoreImage) ADKOpenGLImageView provides an OpenGL solution 47 Summary Making Camera Amazing
- 48. 48 Q + A Making Camera Amazing