Publications

Pre-trained models are not enough: active and lifelong learning is important for long-term visual monitoring of mammals in biodiversity research. Individual identification and attribute prediction with image features from deep neural networks and decoupled decision models applied to elephants and great apes
Paul Bodesheim and Jan Blunk and Matthias Körschens and Clemens-Alexander Brust and Christoph Käding and Joachim Denzler.
Mammalian Biology. 102: pages 875-897. 2022.
[bibtex] [web] [abstract]

Abstract: Animal re-identification based on image data, either recorded manually by photographers or automatically with camera traps, is an important task for ecological studies about biodiversity and conservation that can be highly automatized with algorithms from computer vision and machine learning. However, fixed identification models only trained with standard datasets before their application will quickly reach their limits, especially for long-term monitoring with changing environmental conditions, varying visual appearances of individuals over time that differ a lot from those in the training data, and new occurring individuals that have not been observed before. Hence, we believe that active learning with human-in-the-loop and continuous lifelong learning is important to tackle these challenges and to obtain high-performance recognition systems when dealing with huge amounts of additional data that become available during the application. Our general approach with image features from deep neural networks and decoupled decision models can be applied to many different mammalian species and is perfectly suited for continuous improvements of the recognition systems via lifelong learning. In our identification experiments, we consider four different taxa, namely two elephant species: African forest elephants and Asian elephants, as well as two species of great apes: gorillas and chimpanzees. Going beyond classical re-identification, our decoupled approach can also be used for predicting attributes of individuals such as gender or age using classification or regression methods. Although applicable for small datasets of individuals as well, we argue that even better recognition performance will be achieved by improving decision models gradually via lifelong learning to exploit huge datasets and continuous recordings from long-term applications. We highlight that algorithms for deploying lifelong learning in real observational studies exist and are ready for use. Hence, lifelong learning might become a valuable concept that supports practitioners when analyzing large-scale image data during long-term monitoring of mammals.

Carpe Diem: A Lifelong Learning Tool for Automated Wildlife Surveillance
Clemens-Alexander Brust and Björn Barz and Joachim Denzler.
Computer Science for Biodiversity Workshop (CS4Biodiversity), INFORMATIK 2021. Pages 417-423. 2021.
[bibtex] [pdf]

Self-Supervised Learning from Semantically Imprecise Data
Clemens-Alexander Brust and Björn Barz and Joachim Denzler.
arXiv preprint arXiv:2104.10901. 2021.
[bibtex] [pdf] [abstract]

Investigating the Consistency of Uncertainty Sampling in Deep Active Learning
Niklas Penzel and Christian Reimers and Clemens-Alexander Brust and Joachim Denzler.
DAGM German Conference on Pattern Recognition (DAGM-GCPR). Pages 159-173. 2021.
[bibtex] [pdf] [web] [abstract]

Making Every Label Count: Handling Semantic Imprecision by Integrating Domain Knowledge
Clemens-Alexander Brust and Björn Barz and Joachim Denzler.
International Conference on Pattern Recognition (ICPR). 2020.
[bibtex] [pdf] [abstract]

Active and Incremental Learning with Weak Supervision
Clemens-Alexander Brust and Christoph Käding and Joachim Denzler.
Künstliche Intelligenz (KI). 2020.
[bibtex] [pdf] [abstract]

Automated Polyp Differentiation on Coloscopic Data using Semantic Segmentation with CNNs
Marie Arlt and Jack Peter and Sven Sickert and Clemens-Alexander Brust and Joachim Denzler and Andreas Stallmach.
Endoscopy. 51 (04): pages 4. 2019.
[bibtex] [web] [abstract]

Active Learning for Deep Object Detection
Clemens-Alexander Brust and Christoph Käding and Joachim Denzler.
International Conference on Computer Vision Theory and Applications (VISAPP). Pages 181-190. 2019.
[bibtex] [pdf] [abstract]

Integrating Domain Knowledge: Using Hierarchies to Improve Deep Classifiers
Clemens-Alexander Brust and Joachim Denzler.
Asian Conference on Pattern Recognition (ACPR). 2019.
[bibtex] [pdf] [abstract]

Not just a Matter of Semantics: The Relationship between Visual Similarity and Semantic Similarity
Clemens-Alexander Brust and Joachim Denzler.
DAGM German Conference on Pattern Recognition (DAGM-GCPR). Pages 414-427. 2019.
[bibtex] [pdf] [abstract]

Registration of High Resolution Sar and Optical Satellite Imagery Using Fully Convolutional Networks
Stefan Hoffmann and Clemens-Alexander Brust and Maha Shadaydeh and Joachim Denzler.
International Geoscience and Remote Sensing Symposium (IGARSS). Pages 5152-5155. 2019.
[bibtex] [pdf] [abstract]

Keeping the Human in the Loop: Towards Automatic Visual Monitoring in Biodiversity Research
Joachim Denzler and Christoph Käding and Clemens-Alexander Brust.
International Conference on Ecological Informatics (ICEI). Pages 16. 2018.
[bibtex] [abstract]

Abstract: More and more methods in the area of biodiversity research grounds upon new opportunities arising from modern sensing devices that in principle make it possible to continuously record sensor data from the environment. However, these opportunities allow easy recording of huge amount of data, while its evaluation is difficult, if not impossible due to the enormous effort of manual inspection by the researchers. At the same time, we observe impressive results in computer vision and machine learning that are based on two major developments: firstly, the increased performance of hardware together with the advent of powerful graphical processing units applied in scientific computing. Secondly, the huge amount of, in part, annotated image data provided by today's generation of Facebook and Twitter users that are available easily over databases (e.g., Flickr) and/or search engines. However, for biodiversity applications appropriate data bases of annotated images are still missing. In this presentation we discuss already available methods from computer vision and machine learning together with upcoming challenges in automatic monitoring in biodiversity research. We argue that the key element towards success of any automatic method is the possibility to keep the human in the loop - either for correcting errors and improving the system's quality over time, for providing annotation data at moderate effort, or for acceptance and validation reasons. Thus, we summarize already existing techniques from active and life-long learning together with the enormous developments in automatic visual recognition during the past years. In addition, to allow detection of the unexpected such an automatic system must be capable to find anomalies or novel events in the data. We discuss a generic framework for automatic monitoring in biodiversity research which is the result of collaboration between computer scientists and ecologists of the past years. The key ingredients of such a framework are initial, generic classifier, for example, powerful deep learning architectures, active learning to reduce costly annotation effort by experts, fine-grained recognition to differentiate between visually very similar species, and efficient incremental update of the classifier's model over time. For most of these challenges, we present initial solutions in sample applications. The results comprise the automatic evaluation of images from camera traps, attribute estimation for species, as well as monitoring in-situ data in environmental science. Overall, we like to demonstrate the potentials and open issues in bringing together computer scientists and ecologist to open new research directions for either area.

Dataless Black-Box Model Comparison
Christoph Theiß and Clemens-Alexander Brust and Joachim Denzler.
Pattern Recognition and Image Analysis. Advances in Mathematical Theory and Applications (PRIA). 28 (4): pages 676-683. 2018. (also published at ICPRAI 2018)
[bibtex] [abstract]

Towards Automated Visual Monitoring of Individual Gorillas in the Wild
Brust, Clemens-Alexander and Burghardt, Tilo and Groenenberg, Milou and Käding, Christoph and Kühl, Hjalmar and Manguette, Marie and Denzler, Joachim.
ICCV Workshop on Visual Wildlife Monitoring (ICCV-WS). Pages 2820-2830. 2017.
[bibtex] [pdf] [abstract]

You Have To Look More Than Once: Active and Continuous Exploration using YOLO
Clemens-Alexander Brust and Christoph Käding and Joachim Denzler.
CVPR Workshop on Continuous and Open-Set Learning (CVPR-WS). 2017. Poster presentation and extended abstract
[bibtex] [abstract]

Neither Quick Nor Proper -- Evaluation of QuickProp for Learning Deep Neural Networks
Clemens-Alexander Brust and Sven Sickert and Marcel Simon and Erik Rodner and Joachim Denzler. 2016. Technical Report TR-FSU-INF-CV-2016-01
[bibtex] [pdf] [abstract]

Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding
Clemens-Alexander Brust and Sven Sickert and Marcel Simon and Erik Rodner and Joachim Denzler.
International Conference on Computer Vision Theory and Applications (VISAPP). Pages 510-517. 2015.
[bibtex] [pdf] [abstract]

Efficient Convolutional Patch Networks for Scene Understanding
Clemens-Alexander Brust and Sven Sickert and Marcel Simon and Erik Rodner and Joachim Denzler.
CVPR Workshop on Scene Understanding (CVPR-WS). 2015. Poster presentation and extended abstract
[bibtex] [pdf] [abstract]