January 25, 2021 THETA Plug-in orientation

Tilt Your THETA! Using Orientation to Enable THETA Settings Without Using A Smartphone

Tilt Your THETA! Using Orientation to Enable THETA Settings Without Using A Smartphone

Introduction

Original article can be found here.

We have a camera called RICOH THETA that can take pictures in 360 degrees in all directions. The RICOH THETA V and RICOH THETA Z1 models use Android as the OS. You can customize THETA just like developing an Android app. The customization function is called a “plug-in” (see the end of this article for details). In this article, we will look at operating a RICOH THETA Z1 by using the orientation of the camera. I think a RICOH THETA V can be programmed to do something similar, but I focused on using a Z1 because you can use the OLED. It would be great if you use these techniques with a RICOH THETA V or create your own version for RICOH THETA Z1 saying, “I will do this myself.”

This is how it looks in action.

https://www.youtube.com/watch?v=5kFv-7Cc3h8&feature=youtu.be

This THETA plug-in is now available in the THETA Plug-in Store. It’s called the Change via Tilt plug-in. Please download it and try it out!

Specifications of What We Made

Here are the things you can do with this plug-in. Since the goal was to “not need a smartphone,” we can set everything!

  • Switching shooting mode (image / video)
  • Switching the exposure mode (Auto / Av / Tv / Iso / Manual)
  • Shooting settings for each exposure option
  • (Aperture, shutter speed, ISO sensitivity, white balance, option settings for still image auto mode [NR / DR Comp / HDR / Hh HDR])
  • Switching the white balance specification method (preset / color temperature)
  • Switching the file storage format (JPEG / RAW +) when shooting still images
  • Switching the self-timer on / off
  • Switching continuous shooting options (Off / Time Shift / Interval / Interval Composite)

The plug-in also handles the following items:

  • The settings made with this plugin will be saved when the plugin is closed and will be restored when the plugin is started again. It can also be used as a “super MY setting.”
  • This plug-in is compatible with remote controllers (HID devices that send out the “Vol +” key code).

I will explain what you see in the video, and I will explain the behavior of this plug-in item by item. Basically, the operations of the plug-in follow these rules:

  • Do not change parameters without pressing a button
    • (“JPEG <-> RAW + switching” and “selecting the white balance specification method” are exceptions, but we have taken care that they are not changed easily nor can be confirmed with one action when looking at the display.)
  • Items that are not operated frequently cannot be changed unless they are turned “upside down.”

Since these are our specifications, there may be some points that do not suit your favorite workflow. If you really want to change it, the source code is available, so please customize it yourself.

How to Select Changeable Items by Tilting

Based on the orientation of an upright THETA, the display items in the double line frame on the right side of the OLED can be switched by tilting 45 degrees or more and then returning to the original upright orientation. Since it is not an elaborate mechanism (it does not have hysteresis - magnetic movement - in the orientation assessment), it will switch if the movement is close to 45 degrees, but it should be accurate enough for practical use.

You can select which item to display depending on the direction of the tilt. The correspondence between the tilt direction and which display item is changed as follows. Also, a tilt guide is displayed on the OLED, so you won’t get lost.

changeable

Button Operations

If the THETA is oriented in directions other than upside down, you can change the value of the item selected described above. Since I wanted to support continuous changes by pressing and holding the button, the Mode button is assigned to “Select exposure program.” (Please note: Pressing and holding down the Mode button is reserved for ending plug-in operation.)

Button operation while upside-down (tilted at 135 degrees or more compared to straight upright is considered upside down) preserves normal camera operation techniques without running the plug-in. The behavior of Mode and Fn is the same.

Since this plug-in does not need to use Wi-Fi, I assigned “Switch to special shooting” to the Wi-Fi button.

buttons

Changing White Balance

White balance (preset / color temperature) can be switched only when white balance is displayed using the tilt operation described above.

I don’t expect users to change white balance frequently, so I’ve made it difficult to change by switching it only when it’s completely upside down. (However, it is recommended to check visually before shooting.)

white-balance

Changing Still Image File Format

The file format can be selected by quickly twisting to the right or left based while being held upright. “Twist quickly” is described as “rotating 90 degrees or more in 200 milliseconds.” This may be hard to visualize without using it, but it’s easy once you try it. However, it’s a small movement so switching does not happen too easily.

format

OLED Display

The icons for the OLED display are as follows. There’s not a lot of extra room! I don’t want to add more icons beyond this. (I think it matches the style of the RICOH THETA Z1. It looks like a normal camera.

oled

Advanced users might have asked, “TimeShift and the self-timer can be turned on / off independently, but why are they overlapping?” Since Time Shift has priority when both are set to On it is displayed like this.

Some extra details:

  • “In progress” is displayed during time-shift shooting and interval shooting
  • “Elapsed time” is displayed during interval composition
  • “Recording time” is displayed during video recording
  • When exposure is manual, the brightness of the OLED is adjusted according to the set value

details

The last point is just my own opinion. I don’t like the display brightness too be too bright when shooting manually in a dark place, so I made it possible to choose lower levels.

(Even if you operate the aperture, shutter speed, or ISO sensitivity, the brightness of the OLED is changed by one setting every 0.3 steps. The more light is taken into the camera, the darker the OLED, and vice versa.)

Constraints

  • The display order of the options setting (_filter) is determined by the response of the RICOH THETA API
  • It takes time to finish interval shooting and interval composition. After you finish, you can restart taking individual shots, but please wait for a while.
  • The interval shooting settings are fixed at “shooting interval = shortest” and “shooting period = ∞ (= 0)”
  • The setting for interval composition is set to “Save in progress = None” and “Shooting time = 24 hours”
  • Set the self-timer time from the basic RICOH THETA app

How to Use Tilt Operations

SensorManager.getOrientation, a standard Android library which obtains the orientation information is used.

The local coordinate system for general Android smartphones and the local coordinate system for THETA are defined as follows. Be aware of these differences when reading Google documentation.

orientation

And there are the following differences in the range of ψ = azimuth (Azimath / Yaw), θ = gradient (Pitch), φ = rotation (Roll).

Ψ range Θ range Φ range
General smartphone -π~π -π~π -π/2~π/2
THETA -π~π -π/2~π/2 -π~π

For θ and Φ, one of the ranges should be -π / 2 to π / 2 and the other should be -π to π.

For THETA, images are obtained in the format of Equirectangular (the format of the world map in the relationship between the globe and the world map), and the vertical axis of the image shows Pitch. Since this area is -π / 2 to π / 2, we are trying to deal with it. It’s also intuitive and easy to understand.

Since orientation can be obtained from the angle of the 3 axes every moment, just write the code that performs the desired movement according to it. It’s very easy.

Source Code

See the complete project here. A detailed explanation focusing on key points is given below.

File Organization

Since various shooting settings are made using the RICOH THETA API, I created them based on the source code of this article (in Japanese), which already covered most shooting and OLED display settings.

Newly created or newly modified files are as follows.

theta-plugin-switchbot-link\app\src\main
├assets       // Contains OLED display images. There are some new icons, but they look good, so I won't explain them.
└java\com\theta360
 ├pluginapplication
 │ ├model   // THETA Plug-in SDK as-is
 │ ├network // Added a method to HttpConnector.java like before 
 │ ├oled    // Oled.java (basically an OLED drawing library) only
 │ └task    // ChangeFilterTask.java and ChangeFormatTask.java are new
 │          // Fixed GetCameraStatusTask.java, ShutterButtonTask.java, ChangeShutterSpeedTask.java, ChangeWbTask.java
 └tiltui    // Attitude.java is the same as before. It is a convenient class.
            // MainActivity.java and DisplayInfo.java contain the unique items of this plug-in

How to Asses Tilt

Please refer to the links below for information on the following three basic items. All three links are in Japanese.

About getOrientation

Posture class Attitude.java

How to use posture class

Basic Structure of the Program

A thread that operates periodically is created in MainActivity.java, and the following processing is performed. Since the status of the shooting task is acquired periodically by WebAPI, it is operated while taking a rest of 200 milliseconds, so that the internal communication load does not become too high.

   public void drawOledThread() {
        new Thread(new Runnable() {
            @Override
            public void run() {

                //If there is something to do before the loop ...

                //Drawing loop
                while (mFinished == false) {
                    //Status check and drawing
                    try {
                        //Orientation information acquisition & orientation related information update
                        displayInfo.setTiltStatus(attitude.getDegAzimath(), attitude.getDegPitch(), attitude.getDegRoll());

                        //Internal state acquisition
                        new GetCameraStatusTask(mGetCameraStatusCallback).execute();

                        // OLED display
                        displayInfo.displayTiltUI();

                        // Upside down && WB changeable -> Preset <-> CT switching
             ~abbreviation~

                        //Processing when twisting operation is detected
             ~abbreviation~

                        //Execution of button operations
             ~abbreviation~

                        //Sleep so that internal communications load does not get too high
                        Thread.sleep(200);

                    } catch (InterruptedException e) {
                        // Deal with error.
                        e.printStackTrace();
                    } finally {
                        //
                    }

                }
            }
        }).start();
    }

Judging the Orientation

Judging the orientation of the THETA is done by the setTiltStatus method of the Displayinfo class, which is called at the beginning of the above periodic operation thread.

The processing contents are as follows.

   void setTiltStatus(double inYaw, double inPitch, double inRoll) {
        beforDegAzimath = degAzimath;
        degAzimath = inYaw;
        degPitch = inPitch;
        degRoll = inRoll;

        beforTilt = curTilt;
        if ( (-45 <= degPitch) && (degPitch<45) ) {
            if ( (-45<=degRoll) && (degRoll<=45) ) {
                curTilt = TILT_BASE;
            } else if ( (45<=degRoll) && (degRoll<=135) ) {
                curTilt = TILT_RIGHT;
            } else if ( (-135<=degRoll) && (degRoll<=-45) ) {
                curTilt = TILT_LEFT;
            } else if ( ((135<=degRoll) && (degRoll<=180)) ||
                    ( (-180<=degRoll) && (degRoll<=-135) ) ) {
                curTilt = TILT_UPSIDE_DOWN;
                occurUpsideDown=true;
            } else {
                curTilt = TILT_ERROR;
            }
        } else if ( (45 <= degPitch) && (degPitch<=90) ) {
            curTilt = TILT_FRONT;
        } else if ( (-90 <= degPitch) && (degPitch<=-45) ) {
            curTilt = TILT_BACK;
        } else {
            curTilt = TILT_ERROR;
        }

        //Twist event check
        if ( curTilt == TILT_BASE ) {
            double diffAzimath = beforDegAzimath-degAzimath;
            if ( diffAzimath >= TWIST_THRESH ) {
                lastEvent = EVENT_TWIST_L;
            } else if ( diffAzimath <= -1.0*TWIST_THRESH) {
                lastEvent = EVENT_TWIST_R;
            } else {
                //No twist event
            }
        }

        //Tilt event check
        boolean occurChangeEvent = false;
        if ( (curTilt==TILT_BASE)&&(beforTilt!=TILT_BASE) ) {
            occurChangeEvent = true;

            if (occurUpsideDown) {
                occurUpsideDown=false;
                lastEvent =EVENT_UPSIDE_DOWN;
            } else {
                switch (beforTilt) {
                    case TILT_FRONT :
                        lastEvent =EVENT_FRONT;
                        break;
                    case TILT_BACK :
                        lastEvent =EVENT_BACK;
                        break;
                    case TILT_RIGHT :
                        lastEvent =EVENT_RIGHT;
                        break;
                    case TILT_LEFT :
                        lastEvent =EVENT_LEFT;
                        break;
                    default:
                        //No processing: It shouldn't happen but just in case
                        break;
                }
            }

            // Changeable parameter transition
            ~abbreviation~

        }

    }

First, after determining the current orientation, the event is set according to the change from the previous orientation in the periodic motion. The process that follows this process only behaves in response to the event.

It’s a hassle to write, but it’s not difficult.

Common Structure of Tasks for Setting Shooting

With this plug-in, you can set various shooting settings, and 11 types of tasks are prepared for each setting item. I wanted to have a common Callback process after each task was processed, so I am using a mechanism called Consumer on Android. It is not full of callbacks and can be described clearly.

Conclusion

Partially because of the almost stick shape of a RICOH THETAe, I think we were able to expand functions very well.

I think there are quite a lot of RICOH THETA Z1 users who want to make shooting settings without a smartphone. Remember, it is available in the THETA Plug-in Store. It’s called the Change via Tilt plug-in. Please give it a tilt!