ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Introduction of Android Camera1

?Download as PPTX, PDF?
?
Booch Lin
Booch Lin

more detail blog:http://vinta.ws/booch/?p=20 ref: http://blog.chinaunix.net/uid-28416178-id-3446034.html http://blog.csdn.net/llping2011/article/details/9372799 http://blog.163.com/shawpin@126/blog/static/116663752201092394147937/ http://www.eoeandroid.com/blog-10407-5106.html

1 of 24
Downloaded 129 times
Introduction for Camera Heaton
? Camera Framework ? Camera Architecture ? From java to hardware: Camera.open flow ? startPreview() ? From hardware to Java: Jpeg Callback flow Contents
Camera Framework
Camera Architecture Camera JNI : frameworks/base/core/jni/android_hardware_Camera.cpp (libandroid_runtime.so) Camera client: Header files: frameworksavincludecamera CPP files: frameworksavcamera (libcamera_client.so) Camera Service: frameworksavservicescameralibcameraservice (libcameraservice.so)
Architecture
Application Framework/SDK Java Classes Exposed to the application for interacting with the hardware frameworks/base/core/java/androi d/hardware JNI Glue Code between Java Classes and Native Classes frameworks/base/core/jni/android _hardware_Camera.cpp Native Framework Native counterpart of the Java Camera Classes. Manages all Binder Interactions frameworks/av/camera/Camera.c pp IPC Binder 3 Binder Interfaces ICameraService and Icamera from Application to Framework. ICameraClient for callbacks into the Application frameworks/av/camera Camera Service The Camera Service that manages permissions and lifecycle of the Camera Devices frameworks/av/services/camera/li bcameraservice/CameraService.c pp HAL Interface Hardware Interface. platform/hardware/libhardware/inc lude HAL Implementation Hardware specific implementations. Depends on host processor, ISP and Sensor platform/hardware/<vendor>/<plat form> Kernel drivers
HAL1
Camera keynote ¡ñ Camera uses binderized classes ¡ð ICamera -- proxy to camera hardware ¡ð ICameraClient -- receives callbacks ¡ð ICameraService -- creates and controls Icamera ¡ñ ICameraService, like ICamera and ICameraClient, are binder interfaces(proxy) ¡ñ Defined in frameworks/av/: ¡ð include/camera/ICamera*.h ¡ð camera/ICamera*.cpp ¡ð service/camera/libcameraservice/CameraService.cpp ¡ñ Camera class: ¡ð is a BnCameraClient ¡ð contains an Icamera
From top to bottom flow: Camera open case
Step 1: open() (1)frameworks/base/core/java/android/hardware/Camera.java public class Camera { public static Camera open(int cameraId) { return new Camera(cameraId); } Camera(int cameraId) { ................. Looper looper; if ((looper = Looper.myLooper()) != null) { mEventHandler = new EventHandler(this, looper); } else if ((looper = Looper.getMainLooper()) != null) { mEventHandler = new EventHandler(this, looper); } else { mEventHandler = null; } native_setup(new WeakReference<Camera>(this), cameraId); } }
Step 2:Use JNI to call Native function (2) frameworks/base/core/jni/android_hardware_Camera.cpp static JNINativeMethod camMethods[] = { { ¡°native_setup¡±, ¡°(Ljava/lang/Object;I)V¡±, (void*)android_hardware_Camera_native_setup }, /*use Camera.cpp to connect CameraService*/ ¡­... };
Step 3: call connect , set call back listener (3) framework/av/camera/Camera.cpp static void android_hardware_Camera_native_setup(JNIEnv *env, jobject jobject weak_this, jint cameraId) { sp<Camera> camera = Camera::connect(cameraId); ¡­ jclass clazz = env->GetObjectClass(thiz); // We use a weak reference so the Camera object can be garbage colle // The reference is only used as a proxy for callbacks. sp<JNICameraContext> context = new JNICameraContext(env, weak_th camera); context->incStrong(thiz); camera->setListener(context);//CameraListener. // ICamera and encapsulates it into a Camera object, which it returns to JNI env->SetIntField(thiz, fields.context, (int)context.get()); }
Step 4: get Camera from BpCameraService (4) framework/av/camera/CameraBase.cpp sp<Camera> Camera::connect(int cameraId) { ALOGV("connect"); sp<Camera> c = new Camera();//BnCameraClient const sp<ICameraService>& cs = getCameraService();//return BpCameraService if (cs != 0) {//Used for processing all kinds of events c->mCamera = cs->connect(c, cameraId);//return BpCamera } if (c->mCamera != 0) { c->mCamera->asBinder()->linkToDeath(c); c->mStatus = NO_ERROR; } else { c.clear(); } return c; }
Step 5: Use IPC binder between BnCameraService and BpCameraService (5)framework/av/service/camera/libcameraservice/ICameraService.cpp class BpCameraService: public BpInterface<ICameraService> { public: // connect to camera service virtual sp<ICamera> connect(const sp<ICameraClient>& cameraClient, int cameraId) { Parcel data, reply; data.writeInterfaceToken(ICameraService::getInterfaceDescriptor()); data.writeStrongBinder(cameraClient->asBinder());//ÞD“Q³ÉIBinderÀà ÐÍBpCameraClient data.writeInt32(cameraId); remote()->transact(BnCameraService::CONNECT, data, &reply); return interface_cast<ICamera>(reply.readStrongBinder());//BpCamera } };
Step 5: Use IPC binder between BnCameraService and BpCameraService (5)framework/av/service/camera/libcameraservice/ICameraService.cpp status_t BnCameraService::onTransact( uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags) { switch(code) { case CONNECT: { CHECK_INTERFACE(ICameraService, data, reply);// sp<ICameraClient> cameraClient = interface_cast<ICameraClient>(data.readStrongBinder()); //get BpCameraClient sp<ICamera> camera = connect(cameraClient, data.readInt32());// //return Client À^³ÐBnCamera reply->writeStrongBinder(camera->asBinder()); //¨@ÞD³ÉBpCamera Œ‘»Øreply return NO_ERROR; } break; } }
(6)framerwork/av/service/camera/libcameraservice/CameraService.cpp sp<ICamera> CameraService::connect( const sp<ICameraClient>& cameraClient, int cameraId) { ¡­.. int deviceVersion; if (mModule->common.module_api_version == CAMERA_MODULE_API_VERSION_2_0) { deviceVersion = info.device_version; } else { deviceVersion = CAMERA_DEVICE_API_VERSION_1_0; } /*¸ù“þHAL²»Í¬API„“½¨²»Í¬µÄClient instance after 4.2 version*/ switch(deviceVersion) { case CAMERA_DEVICE_API_VERSION_1_0: client = new CameraClient(this, cameraClient, clientPackageName, cameraId, facing, callingPid, clientUid, getpid()); break; case CAMERA_DEVICE_API_VERSION_2_0: case CAMERA_DEVICE_API_VERSION_2_1: case CAMERA_DEVICE_API_VERSION_3_0: client = new Camera2Client(this, cameraClient, clientPackageName, cameraId, facing, callingPid, clientUid, getpid(), deviceVersion); break; } /*³õʼ»¯camera_module_t *module*/ if (client->initialize(mModule) != OK) { return NULL; } ¡­.... return client;/*×îºó·µ»Ø*/ } Step 6: instance CameraClient
Initialize CameraClient (8)HAL 1 Case..CameraClient status_t CameraClient::initialize(camera_module_t *module { ¡­.... mHardware = new CameraHardwareInterface(camera_device_name); res = mHardware->initialize(&module->common); ¡­.... mHardware->setCallbacks(notifyCallback, dataCallback, dataCallbackTimestamp, (void *)mCameraId); ¡­.. }
Initialize Camera2Client (8)HAL 2or3 Case..Camera2Client status_t Camera2Client::initialize(camera_module_t *module) { mDevice = CameraDeviceFactory::createDevice(cameraId); //Camera2Device ¡­... mDevice->initialize(module); ¡­.. mDevice->setNotifyCallback(this); return OK; }
You can call medial server method by IPC
Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener 1. call back from hardward , flow start from camera client ex:jpeg call back void CameraClient::dataCallback(int32_t msgType, const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata, void* user) { CameraClient* client = static_cast<CameraClient*>(getClientFromCookie(user)); switch (msgType & ~CAMERA_MSG_PREVIEW_METADATA) { ¡­.... case CAMERA_MSG_COMPRESSED_IMAGE: client->handleCompressedPicture(dataPtr); Break; ¡­..... } } ICameraClientJNIEnv AP Camera xxxListener 2. use ipc binder ¡°bpcameraclient , bncameraclient in IcameraClient to send data , and implement in Camera.cpp void CameraClient::handleCompressedPicture(const sp<IMemory>& mem) { sp<ICameraClient> c = mRemoteCallback; mLock.unlock(); if (c != 0) { c->dataCallback(CAMERA_MSG_COMPRESSED_IMAGE, mem, NULL); } 12
Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener 3. use ipc binder ¡°bpcameraclient , bncameraclient in IcameraClient to send data , and implement in Camera.cpp void Camera::dataCallback(int32_t msgType, const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata) { sp<CameraListener> listener; { Mutex::Autolock _l(mLock); listener = mListener; } if (listener != NULL) { listener->postData(msgType, dataPtr, metadata); } } ICameraClientJNIEnv AP Camera xxxListener 3
Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener class CameraListener: virtual public RefBase { public: virtual void notify(int32_t msgType, int32_t ext1, int32_t ext2) = 0; virtual void postData(int32_t msgType, const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata) = 0; virtual void postDataTimestamp(nsecs_t timestamp, int32_t msgType, const sp<IMemory>& dataPtr) = 0; }; ICameraClientJNIEnv Note: msgType enum values and data structures, ex: ¡°camera_frame_metadata_t¡± are defined in ¡°systemcoreincludesystemcamera.h¡±. AP Camera xxxListener
Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener 4.Implement cameraListener post data in android_haradware_camera.cpp void JNICameraContext::postData(int32_t msgType, const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata) { ¡­....... JNIEnv *env = AndroidRuntime::getJNIEnv(); ..... int32_t dataMsgType = msgType & ~CAMERA_MSG_PREVIEW_METADATA; switch (dataMsgType) { ¡­.... case CAMERA_MSG_RAW_IMAGE: ALOGV("rawCallback"); if (mRawImageCallbackBuffers.isEmpty()) { env->CallStaticVoidMethod(mCameraJClass, fields.post_event, mCameraJObjectWeak, dataMsgType, 0, 0, NULL); } else { copyAndPost(env, dataPtr, dataMsgType); } Break; } ICameraClientJNIEnv AP Camera xxxListener 5.post image data to Java by JNI void JNICameraContext::copyAndPost(JNIEnv* env, const sp<IMemory>& dataPtr, int msgType) { obj = env->NewByteArray(size); env->CallStaticVoidMethod(mCameraJClass, fields.post_event, mCameraJObjectWeak, msgType, 0, 0, obj); } //jclass clazz = env->FindClass("android/hardware/Camera"); fields.post_event = env->GetStaticMethodID(clazz, "postEventFromNative", "(Ljava/lang/Object;IIILjava/lang/Object;)V"); 45
Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener 5. Call back to ap level :Framework/base/core/java/android/hardware/camera.java private class EventHandler extends Handler { @Override public void handleMessage(Message msg) { switch(msg.what) { case CAMERA_MSG_COMPRESSED_IMAGE: if (mJpegCallback != null) { mJpegCallback.onPictureTaken((byte[])msg.obj, mCamera); } Return; } ICameraClientJNIEnv AP Camera xxxListener 6

More Related Content

Introduction of Android Camera1

  • 2. ? Camera Framework ? Camera Architecture ? From java to hardware: Camera.open flow ? startPreview() ? From hardware to Java: Jpeg Callback flow Contents
  • 4. Camera Architecture Camera JNI : frameworks/base/core/jni/android_hardware_Camera.cpp (libandroid_runtime.so) Camera client: Header files: frameworksavincludecamera CPP files: frameworksavcamera (libcamera_client.so) Camera Service: frameworksavservicescameralibcameraservice (libcameraservice.so)
  • 6. Application Framework/SDK Java Classes Exposed to the application for interacting with the hardware frameworks/base/core/java/androi d/hardware JNI Glue Code between Java Classes and Native Classes frameworks/base/core/jni/android _hardware_Camera.cpp Native Framework Native counterpart of the Java Camera Classes. Manages all Binder Interactions frameworks/av/camera/Camera.c pp IPC Binder 3 Binder Interfaces ICameraService and Icamera from Application to Framework. ICameraClient for callbacks into the Application frameworks/av/camera Camera Service The Camera Service that manages permissions and lifecycle of the Camera Devices frameworks/av/services/camera/li bcameraservice/CameraService.c pp HAL Interface Hardware Interface. platform/hardware/libhardware/inc lude HAL Implementation Hardware specific implementations. Depends on host processor, ISP and Sensor platform/hardware/<vendor>/<plat form> Kernel drivers
  • 8. Camera keynote ¡ñ Camera uses binderized classes ¡ð ICamera -- proxy to camera hardware ¡ð ICameraClient -- receives callbacks ¡ð ICameraService -- creates and controls Icamera ¡ñ ICameraService, like ICamera and ICameraClient, are binder interfaces(proxy) ¡ñ Defined in frameworks/av/: ¡ð include/camera/ICamera*.h ¡ð camera/ICamera*.cpp ¡ð service/camera/libcameraservice/CameraService.cpp ¡ñ Camera class: ¡ð is a BnCameraClient ¡ð contains an Icamera
  • 9. From top to bottom flow: Camera open case
  • 10. Step 1: open() (1)frameworks/base/core/java/android/hardware/Camera.java public class Camera { public static Camera open(int cameraId) { return new Camera(cameraId); } Camera(int cameraId) { ................. Looper looper; if ((looper = Looper.myLooper()) != null) { mEventHandler = new EventHandler(this, looper); } else if ((looper = Looper.getMainLooper()) != null) { mEventHandler = new EventHandler(this, looper); } else { mEventHandler = null; } native_setup(new WeakReference<Camera>(this), cameraId); } }
  • 11. Step 2:Use JNI to call Native function (2) frameworks/base/core/jni/android_hardware_Camera.cpp static JNINativeMethod camMethods[] = { { ¡°native_setup¡±, ¡°(Ljava/lang/Object;I)V¡±, (void*)android_hardware_Camera_native_setup }, /*use Camera.cpp to connect CameraService*/ ¡­... };
  • 12. Step 3: call connect , set call back listener (3) framework/av/camera/Camera.cpp static void android_hardware_Camera_native_setup(JNIEnv *env, jobject jobject weak_this, jint cameraId) { sp<Camera> camera = Camera::connect(cameraId); ¡­ jclass clazz = env->GetObjectClass(thiz); // We use a weak reference so the Camera object can be garbage colle // The reference is only used as a proxy for callbacks. sp<JNICameraContext> context = new JNICameraContext(env, weak_th camera); context->incStrong(thiz); camera->setListener(context);//CameraListener. // ICamera and encapsulates it into a Camera object, which it returns to JNI env->SetIntField(thiz, fields.context, (int)context.get()); }
  • 13. Step 4: get Camera from BpCameraService (4) framework/av/camera/CameraBase.cpp sp<Camera> Camera::connect(int cameraId) { ALOGV("connect"); sp<Camera> c = new Camera();//BnCameraClient const sp<ICameraService>& cs = getCameraService();//return BpCameraService if (cs != 0) {//Used for processing all kinds of events c->mCamera = cs->connect(c, cameraId);//return BpCamera } if (c->mCamera != 0) { c->mCamera->asBinder()->linkToDeath(c); c->mStatus = NO_ERROR; } else { c.clear(); } return c; }
  • 14. Step 5: Use IPC binder between BnCameraService and BpCameraService (5)framework/av/service/camera/libcameraservice/ICameraService.cpp class BpCameraService: public BpInterface<ICameraService> { public: // connect to camera service virtual sp<ICamera> connect(const sp<ICameraClient>& cameraClient, int cameraId) { Parcel data, reply; data.writeInterfaceToken(ICameraService::getInterfaceDescriptor()); data.writeStrongBinder(cameraClient->asBinder());//ÞD“Q³ÉIBinderÀà ÐÍBpCameraClient data.writeInt32(cameraId); remote()->transact(BnCameraService::CONNECT, data, &reply); return interface_cast<ICamera>(reply.readStrongBinder());//BpCamera } };
  • 15. Step 5: Use IPC binder between BnCameraService and BpCameraService (5)framework/av/service/camera/libcameraservice/ICameraService.cpp status_t BnCameraService::onTransact( uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags) { switch(code) { case CONNECT: { CHECK_INTERFACE(ICameraService, data, reply);// sp<ICameraClient> cameraClient = interface_cast<ICameraClient>(data.readStrongBinder()); //get BpCameraClient sp<ICamera> camera = connect(cameraClient, data.readInt32());// //return Client À^³ÐBnCamera reply->writeStrongBinder(camera->asBinder()); //¨@ÞD³ÉBpCamera Œ‘»Øreply return NO_ERROR; } break; } }
  • 16. (6)framerwork/av/service/camera/libcameraservice/CameraService.cpp sp<ICamera> CameraService::connect( const sp<ICameraClient>& cameraClient, int cameraId) { ¡­.. int deviceVersion; if (mModule->common.module_api_version == CAMERA_MODULE_API_VERSION_2_0) { deviceVersion = info.device_version; } else { deviceVersion = CAMERA_DEVICE_API_VERSION_1_0; } /*¸ù“þHAL²»Í¬API„“½¨²»Í¬µÄClient instance after 4.2 version*/ switch(deviceVersion) { case CAMERA_DEVICE_API_VERSION_1_0: client = new CameraClient(this, cameraClient, clientPackageName, cameraId, facing, callingPid, clientUid, getpid()); break; case CAMERA_DEVICE_API_VERSION_2_0: case CAMERA_DEVICE_API_VERSION_2_1: case CAMERA_DEVICE_API_VERSION_3_0: client = new Camera2Client(this, cameraClient, clientPackageName, cameraId, facing, callingPid, clientUid, getpid(), deviceVersion); break; } /*³õʼ»¯camera_module_t *module*/ if (client->initialize(mModule) != OK) { return NULL; } ¡­.... return client;/*×îºó·µ»Ø*/ } Step 6: instance CameraClient
  • 17. Initialize CameraClient (8)HAL 1 Case..CameraClient status_t CameraClient::initialize(camera_module_t *module { ¡­.... mHardware = new CameraHardwareInterface(camera_device_name); res = mHardware->initialize(&module->common); ¡­.... mHardware->setCallbacks(notifyCallback, dataCallback, dataCallbackTimestamp, (void *)mCameraId); ¡­.. }
  • 18. Initialize Camera2Client (8)HAL 2or3 Case..Camera2Client status_t Camera2Client::initialize(camera_module_t *module) { mDevice = CameraDeviceFactory::createDevice(cameraId); //Camera2Device ¡­... mDevice->initialize(module); ¡­.. mDevice->setNotifyCallback(this); return OK; }
  • 19. You can call medial server method by IPC
  • 20. Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener 1. call back from hardward , flow start from camera client ex:jpeg call back void CameraClient::dataCallback(int32_t msgType, const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata, void* user) { CameraClient* client = static_cast<CameraClient*>(getClientFromCookie(user)); switch (msgType & ~CAMERA_MSG_PREVIEW_METADATA) { ¡­.... case CAMERA_MSG_COMPRESSED_IMAGE: client->handleCompressedPicture(dataPtr); Break; ¡­..... } } ICameraClientJNIEnv AP Camera xxxListener 2. use ipc binder ¡°bpcameraclient , bncameraclient in IcameraClient to send data , and implement in Camera.cpp void CameraClient::handleCompressedPicture(const sp<IMemory>& mem) { sp<ICameraClient> c = mRemoteCallback; mLock.unlock(); if (c != 0) { c->dataCallback(CAMERA_MSG_COMPRESSED_IMAGE, mem, NULL); } 12
  • 21. Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener 3. use ipc binder ¡°bpcameraclient , bncameraclient in IcameraClient to send data , and implement in Camera.cpp void Camera::dataCallback(int32_t msgType, const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata) { sp<CameraListener> listener; { Mutex::Autolock _l(mLock); listener = mListener; } if (listener != NULL) { listener->postData(msgType, dataPtr, metadata); } } ICameraClientJNIEnv AP Camera xxxListener 3
  • 22. Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener class CameraListener: virtual public RefBase { public: virtual void notify(int32_t msgType, int32_t ext1, int32_t ext2) = 0; virtual void postData(int32_t msgType, const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata) = 0; virtual void postDataTimestamp(nsecs_t timestamp, int32_t msgType, const sp<IMemory>& dataPtr) = 0; }; ICameraClientJNIEnv Note: msgType enum values and data structures, ex: ¡°camera_frame_metadata_t¡± are defined in ¡°systemcoreincludesystemcamera.h¡±. AP Camera xxxListener
  • 23. Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener 4.Implement cameraListener post data in android_haradware_camera.cpp void JNICameraContext::postData(int32_t msgType, const sp<IMemory>& dataPtr, camera_frame_metadata_t *metadata) { ¡­....... JNIEnv *env = AndroidRuntime::getJNIEnv(); ..... int32_t dataMsgType = msgType & ~CAMERA_MSG_PREVIEW_METADATA; switch (dataMsgType) { ¡­.... case CAMERA_MSG_RAW_IMAGE: ALOGV("rawCallback"); if (mRawImageCallbackBuffers.isEmpty()) { env->CallStaticVoidMethod(mCameraJClass, fields.post_event, mCameraJObjectWeak, dataMsgType, 0, 0, NULL); } else { copyAndPost(env, dataPtr, dataMsgType); } Break; } ICameraClientJNIEnv AP Camera xxxListener 5.post image data to Java by JNI void JNICameraContext::copyAndPost(JNIEnv* env, const sp<IMemory>& dataPtr, int msgType) { obj = env->NewByteArray(size); env->CallStaticVoidMethod(mCameraJClass, fields.post_event, mCameraJObjectWeak, msgType, 0, 0, obj); } //jclass clazz = env->FindClass("android/hardware/Camera"); fields.post_event = env->GetStaticMethodID(clazz, "postEventFromNative", "(Ljava/lang/Object;IIILjava/lang/Object;)V"); 45
  • 24. Callback Interfaces Java Framework Camera JNI Client Camera CameraService CameraLinstener 5. Call back to ap level :Framework/base/core/java/android/hardware/camera.java private class EventHandler extends Handler { @Override public void handleMessage(Message msg) { switch(msg.what) { case CAMERA_MSG_COMPRESSED_IMAGE: if (mJpegCallback != null) { mJpegCallback.onPictureTaken((byte[])msg.obj, mCamera); } Return; } ICameraClientJNIEnv AP Camera xxxListener 6