16. One-Minute Demos

???:

• Real time driver fatigue detection
• Dashcam/laptop
• Algorithms:
  • Gaussian blur
  • Histogram of oriented gradients (HOG)
  • SVM
  • Percentage eyelid closure (PERCLOS); eye aspect ratio (EAR), mouth aspect ratio (MAR)

???:

• Wildfire hotspot detection
• Smouldering makes detection difficult
• Heavy duty water carrying drone
• Identify, rank, then approximate distance to hotspot
• Drone camera
• Algorithms:
  • Segmentation: threshold, contour, centroids
  • Morphology
  • Distance approximation (camera calibration)

???:

• Same as above
• Algorithms:
  • Gaussian
  • Binary threshold
  • Morphology: opening/erosion/dilation
  • Contour detection
  • Find largest centroids
  • Lucas-Kanade optical flow: is contour expanding
  • Stereo imaging for distance approximation?

???:

• Sign language teaching assistant
• Real time feedback
• Only RGB camera
• Algorithms:
  • Sharpening kernel
  • Convolutional pose machine
  • Thresholding
  • Hand keypoint detection at some point
• 0.6 Hz

???:

• Track moving objects in robot soccer
• Algorithms:
  • Camera calibration
  • Background subtraction
  • Circular Hough transform + unscented Kalman filter for ball tracking
  • CNN object detection for robot detection

???:

• Faster human detection
• Want low power, fast systems
• CNN-based object detection
• Algorithms:
  • YOLOv5
  • Kalman filter
  • Tracker fit model
    • 0:11:54
    • Run Kalman prediction
    • Run object detection
    • Are they close enough (Euclidean)?
      • No - create new tracker
      • Yes - update Kalman with new location
      • Multiple objects within max distance: pick closest one
• Low CNN depth, Kalman to remove flicker/missed detection in frames

???:

• Gorilla head tracking
• Algorithms:
  • Greyscale frame difference: subtraction
  • Binary thresholding
  • Median blur
  • Morphology: open
  • Find centroid
    • Moment
• Single person only

???:

• Wildfire
• Algorithms:
  • Mean greyscaling
  • Gaussian
  • To-zero thresholding
    • x > threshold? x: 0
  • Circular kernel (blurring?)
  • Pick brightest pixel
  • Blackout pixels around the selected pixel; repeat to get next brightest pixel
  • Guess location by assuming ground is flat: know drone position, camera angle

???:

• FPS enemy detection
• Algorithms:
  • YOLOv5 to get bounding box
  • For outlining:
    • Gaussian
    • Morphology: close
    • Morphology: gradient (difference of dilation and erosion)
    • Thresholding: Otsu

???:

• Rehab to free throw detection (basketball)
• Detectron2 pose detection: pyramid network + Mask R-CNN
• Nothing else?

???:

• Chess board framing, move detection
• Lighting can change
• HSV color masking to detect chess board red/blue pieces
• Algorithms:
  • Board framing:
    • HSV color masking to remove background
    • Morphology: open/close
    • Contour finding: find square
  • Chess grid:
    • Canny edge
    • Hough line transform
    • Homography matrix
  • Chess piece:
    • HSV Color masking

???:

• Construction panel quality control
• Detect dimensions, end tolerances
• Algorithms:
  • Processing:
    • Greyscale
    • Blur
    • Mask
    • Canny edge detection
    • Morphology: erosion/dilation
  • Calibration:
    • Find sticker - pixels to mm
  • Find panel:
    • Draw contour
  • Template match: adapted normalized cross-correlation

???:

• Detect/localize pine tree yellow catkins
• Real time
• Algorithms:
  • YOLOv5 catkin detection
  • MeanShift segmentation of depth image
  • Morphology: opening/closing
  • Contour detection Suzuki’s algorithm
  • Shape matching: Hu moment invariants

???:

• Identify cut logs
• YOLACT with custom dataset for instance segmentation

???:

• Othello piece detection
• Algorithms:
  • Gaussian
  • Color thresholding
  • Douglas-Peucker algorithm
  • Image warping: homography matrix

???:

• Quick-time event detection in Detroit: Become Human
• QTE: fast button/gesture prompt
• Algorithms:
  • Hough circle after greyscale, median blue
  • Text detection:
    • Tesseract OCR
    • Frame crop
    • Bicubic interpolation
    • Gaussian
    • Morphology: Erosion
    • Grayscale
    • Thresholding
  • Harris corner detector to detect symbols
    • Not sure how symbol type is identified

???:

• Rubiks cube tracking, photo -> model of state
• Stickerless cubes - edges not well defined (no black border)
• Algorithms:
  • Harr cascade classifier to detect cube - generate bounding box
  • Split into 3x3 grid
  • Median blur
  • Sample center
  • Color thresholding to classify color
    • Use LAB color space - less sensitive to luminance
• Also used: Canny edge detection, Suzuki85, Douglas-Peucker
• Contour detection: OpenCV, Suzuiki85 https://doi.org/10.1016/0734-189X(85)90016-7

???:

• Integral calculator
• Algorithms:
  • Greyscale
  • Gaussian
  • Binary threshold
  • Morphology: erode/dilate
  • Find contours, sorting by x value
  • Tessaract OCR
  • Convert to string
  • Compute integral
• Fails with handwriting, bad with printed, good with screenshots

???:

• Squash tracking in 3D space
• Position of squash ball
• Algorithms:
  • Subtraction from clean plate (different from background modeling?)
  • Morphology: erosion for noise removal
  • Use output as mask
  • Contour detection: group
  • Filter by size, shape
  • Triangulate position using two different cameras
• Triangulation with two cameras

???:

• Automatic projector keystone calibration
• Algorithms:
  • Canny edge detection
  • Hough to detect image edges
  • Ramer-Douglas-Peucker
  • Homography transform

???:

• Pose estimation: squat depth
• Algorithms:
  • Segmentation mask to separate athlete from background
  • MediaPipe pose
  • Use keypoints to estimate squat depth

???:

• Virtual paper piano
• Printout of keyboard
• Algorithms:
  • Keyboard segmentation:
    • Canny edge detection
    • Merged Hough line transform
    • Perspective transform
    • Linear segmentation
  • Fingertip tracking and touch detection
    • MediaPipe
    • Transform to perspective space
• Only works with a single finger

???:

• Robocup object detection - identify weight
• Algorithms:
  • Stereo calibration (estimate parameters)
  • Gaussian
  • Template matching: normalized correlation coefficient
  • Triangulation to estimate depth

???:

• Bicycle detection and camera tracking (with gimbal)
• Algorithms:
  • YOLO v5 to detect bike
  • Pass bounding box to CSRT tracking algorithm (Channel and Spatial Reliability Tracking)

???:

• Stylus input with CV
• Digital sketchpad
• Algorithms:
  • Canny
  • Hough
  • Shi-Tomasi corner detection
  • Morphology: opening
• Stylus has colored tab which gets exposed when pressure is applied to the tip
  • HSV filter

???:

• Predictive motion
• Algorithms:
  • Gaussian blur
  • Morphology dilation/opening
  • Adaptive thresholding
  • Hough circle transform

???:

• Patient rehabilitation monitor; track pose when doing exercises
• Hardcoded reference angles
• Algorithms:
  • CNN
  • Joint angle calculation, comparison thresholding
  • FSM for multistage verification

42:00

???:

• Melanoma detection using smartphone camera
• Identify moles, return measure for border irregularity and color variance
• Algorithms:
  • Increase image contrast
  • Gaussian
  • Greyscale
  • Morphology: closing
  • Adaptive binary thresholding
  • Suzuki-Abe contour
  • Fitzgibbon ellipse fitting

???:

• Nuclei segmentation in breast cancer tissue images
• Algorithms:
  • Segmentation with convolutional autoencoder (U-net)
  • Thresholding
  • Morphology
  • Watershed

???:

• AR sudoku solver
• Overlay solution on paper
• Algorithms:
  • Pre-processsing:
    • Adaptive thresholding
    • Morphology: open/dilate
    • Grey scale
    • Hough to detect horizontal/vertical lines:
      • Remove the lines to be left with numbers only
      • Perspective transform to warp and crop image
  • CNN for digit classification
  • Solve sudoku using backtracking, render added text, then unwarp to overlay on top of input image

???:

• Tumor/tissue detection: generate tissue and bulk region masks
• Algorithms:
  • Morphology: erosion/dilation
  • Contour detection
  • Color thresholding
  • Median blurring

???:

• Drone detection in airports, differentiate between drone and birds
• Algorithms:
  • Double difference w/ sharpen kernel to detect difference
  • Contour extraction (using morphology - close/dilate)
  • Intersection over union tracker
  • Moving average filter
  • Fast fourier transform to extract power spectral density
  • K-means/linear classifier

???:

• Automatic exposure control for robot navigation
• Algorithms:
  • Sobel gradient filter
  • Gradient magnitude
  • Soft percentile derivative: weighted sum of difference between two frames
  • Slice image into equal sections: take median of soft percentile derivatives
    • No way of automatically calculating number of slices

???:

• Punching technique stats
• Track fist velocity, acceleration, elbow angle, fist angle
• Algorithms:
  • Detectron2 retinanet for fist top/side detection
  • Mediapipe pose
  • Non-maximum suppression

???:

• Football penalty ball tracking
• Camera head-on with goal, located behind player
• Detect goal posts and ball
  • Used yellow ball: HSV color range used to detect ball
• Algorithms:
  • Gaussian
  • Morphology: open, close, erode, dilate
  • Thresholding
  • Contours

???:

• Joint tracking: give likeness score computed from joint angles
• D435 camera
• ~0.4x real time speed
• Algorithms:
  • Region-based CNN (R-CNN): instance segmentation (detectron 2)
    • Keypoint/pose detection?
  • Dynamic time warping
  • Linear interpolation (for dealing with varying framerates)
  • Newton’s method (to align video sequences)

???:

• Scrabble board detection
• Continuing code from previous student project
• Algorithms:
  • Greyscale
  • Adaptive thresholding
  • Detect board contours
  • Crop image to contain only board?
  • HSV masking: filter out undesired colors (i.e. non letter-tiles)
  • Morphology: erosion/dilation to remove noise
  • Detect ‘maximally stable extremal regions’
  • Tesseract OCR

???:

• Rugby ball detection
• Static camera, players in frame
• Difficulty: rugby balls not circular
• Algorithms:
  • Gaussian background subtraction
  • Median filter
  • Morphology: close
  • Contour detection
  • Filter:
    • By area
    • Compactness degree: contour area divided by area of best-fit ellipse
    • By ellipse aspect ratio

???:

• Pet detection using dominant color
• Algorithms:
  • Mask-R CNN instance segmentation to detect pets in frame
  • K-means clustering to determine dominant pet colors
• Image thresholding to remove image background?

???:

• Handwritten digit recognition
• Non-linear transform of input features into higher dimensional space so that features are linearly separable
• Cheaper than conventional deep learning
• Nanowire network: randomly? scattered nanowires combine to form network with complex topological structures: junctions between wires act as a form of non-linearity and memory
• Create 3D simulation of wires: do not assume they are 1D lines
• Input: voltage into input electrodes (1 out of 4 edges of a square)
• Output: current from output electrodes (remaining 3 edges of the square)
  • Train only the output layer through regression to make 10 classifiers
• Algorithms:
  • Linear classifier:
    • Moore-Penrose pseudoinverse
    • Singular value decomposition (SVD)
  • Single layer neural network using Tensorflow
    • Softmax activation
    • Catagorical cross-entropy
    • Adam optimizer
  • Nanowires deposition: Euler rotations
  • Junctions model using Stormer-Verlet integration
  • Modified nodal analysis to solve Kirchoff’s circuit laws

???:

• Blackjack simulation: detect card rank and suit
• Top-down images of cards
• Algorithms:
  • Greyscale
  • Gaussian
  • Thresholding
  • Contours: detect cards
  • Morphology, close: use closing to merge close cards into a single group to detect hands (i.e. based on distance)
• Image differencing: match corner to preset images to detect rank/suit

???:

• Motion detection for raster graphics editor
  • i.e. draw using hand gestures
  • Webcam facing user
• Algorithms:
  • Image rectification
  • Classification/localization CNN to crop to hand
  • Instance segmentation CNN to detect keypoints
  • Use relative distance between keypoints as gesture/controls

???:

• Real-time face replacement
• https://learnopencv.com/face-swap-using-opencv-c-python
• Algorithms:
  • Face detection with dlib library
    • HOG + linear SVM face detector?
  • Face alignment: convex hull
  • Delaunay triangulation
  • Texture mapping by using affine warp to map triangles between the two images
  • OpenCV Seamless cloning

???:

• Blood splatter analysis
• Algorithms:
  • Pre-processing:
    • Thresholding
    • Dilation
    • Resizing
• CNN training: ResNet-50

???:

• Paper piano
• Use built-in laptop camera (paper on top of keyboard/trackpad?)
• One finger only
• Algorithms:
  • Adaptive thresholding (with Gaussian mean)
  • Morphology: opening for noise reduction
  • Ramer-Douglas-Peucker for contour detection
  • Finger detection with color thresholding

???:

• Real time number input for timer control using static hand gestures
• Algorithms:
  • MediaPipe hands
  • Gesture classification:
    • Feature-angle thresholding
    • Support vector machine (SVM)
  • Debouncing:
    • https://ieeexplore.ieee.org/abstract/document/8868766
    • Time/frame-delay debouncing

???:

• Risk parameter of ALC injury risk
• Record jumping video
• Depth camera
• Detect 4 risk factors
• Algorithms:
  • Detectron2 keypoint detection

???:

• Hand gesture controlled calculator
• Use vector of each finger as an input bit
  • Thumbs were a special case
• Algorithms:
  • Hand keypoint detection
  • Kalman filter predictions

???:

• Butterfly/moth classification
• Algorithms:
  • Instance segmentation
  • Non-maximum supression
  • Detectron2 to crop
  • resnet18 to classify

???:

• Blood vessel extraction from image
• Algorithms:
  • Binary thresholding
  • Morphology: open/close/erosion
  • CLAHE algorithm to increase contrast
  • Illumination equalization
  • Gaussian
  • Otsu’s threshold: vessel segmentation

???:

• Hand gesture recognition for sign language w/ smartphone camera
• Algorithms:
  • Histogram
  • Histogram backprojection
  • Morphology: closing
  • Inception v3 neural network

???:

• SLAM with monocular video
• Algorithms:
  • Previous and current frame input into ORB keypoint detection
  • FLANN keypoint matching
  • Lowe’s ratio match pruning
  • RANSAC 5-point: generate essential matrix
  • Velocity information for pose recovery
  • Filter spurious transforms
  • Increment by transformations

???:

• Cow teat detection
• Camera under cow
• Algorithms:
  • Blob detection
    • Thermal filter - coldest = teat
    • Filter by circularity, aspect ratio
  • Morphology: erosion/dilation
  • Repeat with different morphology parameters until four teats in valid shape detected

???:

• Visual cue for call detection for pass gesture in basketball?
• Detect gesture from player so that robot shoots ball?
• Algorithms:
  • Body pose estimation with Detectron2
    • Identify largest skeleton in frame
    • Locate wrist
    • Extract subframes around wrist (proportional to size of skeleton)
    • Mediapipe hand pose
    • Measure distances between certain keypoints to detect gesture
    • Threshold
  • Pass threshold pass/fail to FSM

???:

• Sudoku detection/solving
• Algorithms:
  • Gaussian
  • Adaptive Gaussian thresholding
  • Contour filling: identify contours to determine board outline
  • Green’s theorem
  • Warp image to board shape
  • OCR to identify numbers
  • Solve board using backtracking

???:

• Real time NZ sign language detection
• Algorithms:
  • Pre-process CNN input:
    • Binary thresholding
      • Issues: lighting, busy backgrounds
    • Canny edge detection
  • CNN

???:

• Emotion detection of audiences (multiple people)
• Deepface for facial recognition
  • 9 layer NN
• Algorithms:
  • Haar cascade
  • Face frontalization
  • 2D/3D alignment

???:

• Speed limit recognition of NZ street signs
• Detect, then read
• Detection:
  • Color space transform to emphasize red (CIELUV)
  • Gaussian
  • Circle hough transform
• Text
  • Crop to isolate text
  • Filter
  • Threshold: Otsu’s method
  • Tesseract OCR

???:

• Real time dart scoring
• Identify dart throw events
• Algorithms:
  • Create foreground mask?
  • Board detection:
    • Canny edge
    • Hough transform
    • Morphology
    • Contouring
  • Dart detection:
    • Background subtraction
    • Morphology
    • Contouring
  • Keypoint detection with YOLOv4-tiny

???:

• Stereo imaging to find distance/orientation of plane
• Want to require low overlap
  • Crop images (Right edge/left edge: overlap)
  • Create disparity map
  • Normalize values
  • Convert to depth
  • Average values
  • Reproject to 3D

???:

• Emotion recognition through facial expression
• Algorithms:
  • Haar cascades face recognition
  • Facial action coding system
  • Augmentation for classifier training: brightness, rotation, shift
  • Classifier: CNN + softmax

???:

• Measuring body dimensions with depth camera
• Use 3D points to get dimensions
• Didn’t get to algorithms – out of time

???:

• Music sheet reader for the visually impaired
• Use eye tracking to zoom in and pan into the sheet
• Prior research: automatic page turning but no zoom
• Algorithms:
  • Template matching
  • Gaussian
  • Region of interest
  • Gradient orientation pattern (eye tracking)

???:

• Mobile pool ball detection/identification
• Track position of all balls and identify for scoring
• Prior research: permanent setups
• Try use hand-held video
• Algorithms:
  • Gaussian
  • Hough circle transform to mask ball
• Get average color of mask
• Use color to identify ball score
• Low success rate: 38%

???:

• Inventory stocktaker
• Repeated patterns: instance segmentation?
• One phone camera, using flash to try control lighting
• Detects repeated vertical lines to count items
• Algorithms:
  • Countouring
  • Hough lines
  • Angle filter
  • Morphology: erode/dilate
  • Hough lines

???:

• Beer pong score keeper
• Algorithms:
  • Gaussian
  • Ball detection: color mask + blob detection
  • Cups: hough circle transform
• Issues:
  • Top-down only
  • Lighting/shadows
  • Hardcoded size? Distance/lens combination fixed

???:

• Apple detection (for packaging robot)
• Create own dataset
• Algorithms:
  • To remove background:
    • HSV masking
    • Morphology
• Segmentation labeling for training data
• Mask R-CNN (detectron2)

???:

• Face recognition/tracking on multiple subjects without prior training
• Real time
• Store embedding of faces, store vectors and compare (HOG)
• Algorithms:
  • Single shot multibox detector (resnet base)
  • Ensemble of regression trees
  • Embedding creation: ResNet
  • Recognition: euclidean distance + linear embedding search

???:

• Document supermarket receipts
• Product names, cost, total cost
• Use scanner rather than smartphone camera
• Algorithms:
  • Adaptive binaraization: Otsu’s method
  • Morphology: erosion
  • Tesseract OCR

???:

• Antarctic snowstorm classification with CNN
• ResNet-18 model
• Manually label dataset, then artificially grow
• Augment dataset: horizontal flip, crop, resize
• Classification:
  • Random crops from image: classify each and combine
• Algorithms:
  • Cross entropy loss function
  • Stochastic gradient descent
  • Ensemble method to combine

???:

• Cricket ball shot tracker
• Algorithms:
  • Differencing
  • Thresholding
  • Median blur
  • Dilation
  • Contour detection

???:

• Guitar string picking detection
• Algorithms:
  • Fretboard/frets/string detection:
    • Canny edge
    • Hough line
• Pick location: user identifies initial position for template matching, then tracked
• Detecting picks: after velocity goes above threshold, find rapid deceleration

???:

• Hand gestures to control interactive display
• MediaPipe
• ML to recognize gestures using keypoints
• Limited input speed

???:

• Dartboard segmentation
• Single camera
• Algorithms:
  • HSV color mask
  • Morphology: open/close
  • Flood filling
  • Edge segmentation:
    • Canny edge
    • Hough line
    • Centroid calculation
• Point multipliers (thin green/red regions): HSV + morphology
• Scoring regions: flood filling + bitwise operations
• Wedges: canny + hough line (with centroid to determine board center)

???:

• Cricket batting shot classification with pose estimation
• Algorithms:
  • Gaussian
  • Canny edge detection
  • Detectron2 pose estimation
  • SKLearn

???:

• Dirty dishes on kitchen bench
• Prior: Hough circle, Mask-RCNN
• Solution: Mask-RCNN with COCO dataset
• Plates: transfer learning using resnet?
• Motion detection to reduce false positives
• Gaussian
• Background segmentation

???:

• Darts scoreboard identification
• Blur: gaussian blur. HSV
• Dilate/erosion
• TODO
• Mask generation: thresholding
• Regionsegmentation: Hough lines, canny edge
• Motion detection - double differencing
• Flood fill
• Double differencing
• Triangle interior

???:

• Real time mask detection
• OpenCV/numpy face detection
• Keras/tensorflow face mask
• MobileNetV2

???:

• Manufacturing defect detectoin of surgical reamers
• Otsu thresholding
• Hough transform
• Morphology close
• Topographic TODO
• TODO 0:42

???:

• Robocup object detection, SLAM
• Identify cup weights
• Real time area mapping
• Haar cascade classifier
• ORB feature detection
• TODO
• RANSAC
• ORB-SLAM - didn’t work due to low color variance

???:

• Wheelchair docking assistant
• Prevent damage to desk/chair, injuries
• Slow down when close to the desk
• Object detection: instance segmentation (mask R-CNN)
• Distance calculation: stereo camera

???:

• Low cost stereo with two webcams
• Focus on speed: real time
• Two cameras taped together in a box
• Camera calibration
• Stereo rectification
• Disparity calculation

???:

• Determining queue times with CV
• Greyscale
• Detct faces
  • Front-on faces
• Calcluate bounding box
• Centroid
• TODO
• Haar cascade classifier
• Masking

???:

• Measuring growth rates
• Crop/rotate.
• Blur, hsv theshold, erode/dilate
• Create contours of plants from mask
• Exclude contours outside of region of interest
• Find size of contours
• As leaves get closer to camera, visual size increases

???:

• Vehicle lane positioning with semantic segmentation
• If no road markings
• DeepLavV3 semantic segmenation
• Extract road surface, denoise road surface mask
• median blur
• Color thresold
• Morphology
• Canny edge
• TODO
• 0:53

???:

• Climbing tutor: track 3D poses
• Structure from motion
• Perspective n-point
• Locate keypoints: MediaPipe
• Triangulate
• Bundle adjustment: Least squares solver

???:

• Interactive musical webcam
• Volume/pitch depends on hand position
• Single shot detector
• Feature mapping
• Object prediction: CNN
• Object detection: TODO
• TODO

???:

• Real time background subtraction on people with HMD (AR)
• No green screen, single camera, no depth
• YOLACT for instance segmentation
• Mask
• Contour detection
• Thresholding
• TODO