See also my Google Scholar page.

Learning to Refine Object Segments

Pedro O. Pinheiro*, Tsung-Yi Lin*, Ronan Collobert, Piotr Dollár (ECCV 2016 - spotlight)

In this work we propose to augment feedforward nets for object segmentation with a novel top-down refinement approach. The resulting bottom-up/top-down architecture is capable of efficiently generating high-fidelity object masks. Similarly to skip connections, our approach leverages features at all layers of the net. Unlike them, our approach does not attempt to output independent predictions at each layer. Instead, we first output a coarse ‘mask encoding’ in a feedforward pass, then refine this mask encoding in a top-down pass utilizing features at successively lower layers. [pdf] [bib]

A Multipath Network for Object Detection

Sergey Zagoruyko, Adam Lerer, Tsung-Yi Lin, Pedro O. Pinheiro, Sam Gross, Soumith Chintala, Piotr Dollár (BMVC 2016)

In this paper, we test three modifications to the standard Fast R-CNN object detector: (1) skip connections that give the detector access to features at multiple network layers, (2) a foveal structure to exploit object context at multiple object resolutions, and (3) an integral loss function and corresponding network adjustment that improve localization. It improves results over the baseline Fast R-CNN detector with Selective Search by 66% overall. It placed second in both the COCO 2015 detection and segmentation challenges. [pdf] [bib]

Learning to Segment Object Candidates

Pedro O. Pinheiro, Ronan Collobert, Piotr Dollár (NIPS 2015 - spotlight)

In this paper, we propose a new way to generate object proposals, introducing an approach based on a discriminative convolutional network. Our model is trained jointly with two objectives: given an image patch, the first part of the system outputs a class-agnostic segmentation mask, while the second part of the system outputs the likelihood of the patch being centered on a full object. At test time, the model is efficiently applied on the whole test image and generates a set of segmentation masks, each of them being assigned with a corresponding object likelihood score. [pdf][bib]

Phrase-Based Image Captioning

Remi Lebret*, Pedro O. Pinheiro*, Ronan Collobert (ICML 2015)

Generating a novel textual description of an image is an interesting problem that connects computer vision and natural language processing. In this paper, we present a simple model that is able to generate descriptive sentences given a sample image. This model has a strong focus on the syntax of the descriptions. . We train a purely bilinear model that learns a metric between an image representation (generated from a previously trained Convolutional Neural Network) and phrases that are used to described them. [pdf] [bib]

From Image-level to Pixel-level Labeling with Convolutional Networks

Pedro O. Pinheiro, Ronan Collobert (CVPR 2015)

We are interested in inferring object segmentation by leveraging only object class information, and by considering only minimal priors on the object segmentation task. This problem could be viewed as a kind of weakly supervised segmentation task, and naturally fits the Multiple Instance Learning (MIL) framework: every training image is known to have (or not) at least one pixel corresponding to the image class label, and the segmentation task can be rewritten as inferring the pixels belonging to the class of the object (given one image, and its object class). [pdf] [bib]

Recurrent Convolutional Neural Networks for Scene Labeling

Pedro O. Pinheiro, Ronan Collobert (ICML 2014)

The goal of the scene labeling task is to assign a class label to each pixel in an image. To ensure a good visual coherence and a high class accuracy, it is essential for a model to capture long range pixel) label dependencies in images. We propose an approach that consists of a recurrent convolutional neural network which allows us to consider a large input context while limiting the capacity of the model. Contrary to most standard approaches, our method does not rely on any segmentation technique nor any task-specific features. [pdf] [bib]

A Multiscale Numerical Method for the Heterogeneous Cable Equation

Alexandre L. Madureira, Danele Q.M. Madureira, Pedro O. Pinheiro (Neurocomputing 2012)

Several interesting problems in neuroscience are of multiscale type, i.e. possess different temporal and spatial scales that cannot be disregarded. Such characteristics impose severe burden to numerical simulations since the need to resolve small scale features pushes the computational costs to unreasonable levels. We present a numerical method of multiscale type that ameliorates these maladies. As an example we consider the case of a cable equation modeling heterogeneous dendrites. [pdf] [bib]