Recently, I wrote about the CameraX API and discussed how to use it and what were its advantages for building camera-bound applications when compared with Camera2 or, the now deprecated, Camera API in Android.
In fact, if you read my previous article, you’ll know that the main drawback of Camera2 is its complexity. But, if you get past it, you’ll gain access to the most powerful and complete API for all things camera in Android. So, in this post, let’s demystify the Camera2 API and understand its capabilities and advantages.
Overview
Camera2 is the latest Android camera framework API that replaces the deprecated Camera library and provides in-depth controls necessary for more complex use-cases and scenarios.
Since a single Android device can have multiple cameras each camera module is abstracted by a CameraDevice object. Subsequently, each CameraDevice can output more than one stream simultaneously, each optimized for specific use-cases like displaying a preview, taking a photo, video recording, or image analyzing.
Using CameraDevice you can create a CameraCaptureSession that describes all the streams you require. For each stream, the raw data coming in is transformed to an appropriate format associated with the specific output or Surface as is called in the Camera2 framework. Examples of these Surfaces are the SurfaceView and TextureView for previewing, MediaRecorder for videos, ImageReader for image analysis and capture, plus others like MediaCodec and RenderScriptAllocation.
Also important is the fact that each CameraCaptureSession keeps a queue of CaptureRequest objects to feed its associated CameraDevice with the desired camera hardware configuration requested by the developer. This controls the focus, aperture, effects, exposure, white balance, and other capture-specific characteristics. Notice that you can send as many different CaptureRequests as you like during the lifetime of a CameraCaptureSession.
If it sounds complicated, don’t worry, the next code examples will clarify it.
Getting a Camera
Line 1: Getting a CameraManager object, you’ll need it to fetch a CameraDevice.
Line 3: List of camera identifiers available on the device. Each one represents a different camera module. Notice that if the device doesn’t have a camera this list will be empty.
Line 7–13: Iterating over every camera and getting their individual characteristics (Line 8). After it, we use these characteristics to know if its lens is facing back (Line 9). You can get as detailed as you like when choosing a camera id, check the CameraCharacteristics documentation for features you can inquiry on and the CameraMetadata for the possible responses.
Note that in most devices the camera with id "0" is the main back-facing camera.
Line 15–20: We now open the selected camera and once it is opened, we can fetch the CameraDevice object (Line 17).
Creating a Surface
As mentioned earlier you need one or more Surfaces as receivers of the CameraDevice output. In this guide, we’ll use a SurfaceView for previewing and an ImageReader for image analysis. However, before going into the code make sure you add a SurfaceView to your layout.
<SurfaceView
android:id="@+id/surfaceView"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
/>It is important to note that when you are creating a Surface its size is important so, let’s see how to handle it.
Line 1: To retrieve the supported resolutions of the camera we must obtain the StreamConfigurationMap from its characteristics.
Line 2: The actual resolutions are dependent on the ImageFormat desired output, like JPEG, YUV_420_888, NV21, etc. In this example, we use JPEG.
Line 4–10: Iterate over all resolutions and choose the first one that is less than 500x500. You can use whatever condition you need.
Line 11: Create the ImageReader Surface by setting the desired resolution, the ImageFormat expected for the images (it must be the same as the one used in getOutputSizes) and finally we set the maximum number of images the user will want to access at the same time while analyzing (we’ll see how later). It should be as small as possible to limit memory usage.
Line 13: Getting the SurfaceView from the layout.
Line 14–24: All operations on a SurfaceView, including opening a camera session must be done after it has been created. Therefore, we add a callback to the SurfaceHolder inside SurfaceView to know when this happens.
Line 17–20: The SurfaceView size is set to the same size as the first resolution retrieved from getOutputSizes. Otherwise, it will stretch the image to fill the entire SurfaceView. You can also extend SurfaceView as shown here to manage it automatically or use TextureView.
Initiating a Session
Line 1–3: Create an ArrayList of the Surfaces: ImageReader and SurfaceView in this case.
Line 5: Using the CameraDevice gotten when opening the camera (shown previously) call createCaptureSession and pass it the Surfaces and a callback to know when the session has been configured.
Line 8: Create a CaptureRequest and set the configuration of the camera you want to use. There are several templates available depending on your use-case like TEMPLATE_PREVIEW, TEMPLATE_STILL_CAPTURE, TEMPLATE_RECORD, etc. Check CameraDevice documentation for a complete list of templates.
Line 9: Despite the templates, you can be as specific as you like with the configuration. For instance, here I’m setting the flash to be in torch mode (on all the time).
Line 10–11: It is imperative that you link a CaptureRequest to the Surfaces that it serves, since you can use different CaptureRequests for different Surfaces in the same session. Here though, the same CaptureRequest will serve both of our Surfaces.
Line 13: Using setRepeatingRequest the CaptureRequest will be repeated indefinitely until the session is closed or the requests are stopped.
Line 16–17: Shows an example of a single CaptureRequest for a picture that will be delivered to the ImageReader Surface. If you run this code as is, however, the request will occur almost simultaneously with the initial requests from Line 13. This is intended to be connected to an onClick event of a button, for instance.
Line 19–23: Adding a listener to ImageReader in order to receive and process the images from the requests.
Line 20: This shows how you can access the underlying image from the ImageReader to do whatever you need with them, like saving it to permanent storage or analyzing it.
Conclusion
At this point, you may be frustrated about the apparent complexity of the Camera2 API, but take a moment to re-read this post, and soon you’ll find there is a method to the madness and that is actually not that daunting, especially considering the sheer amount of control you are getting over the device’s camera.
That said, I strongly encourage you to take a look at this post I made about the CameraX Jetpack support library, because more often than not it will be more than enough for your use-case and it is much simpler to use.
Let me know in the comments if this helped you and if you have any doubts.
