Francisco Ibarrola

Over the past few years, rapid developments in AI have resulted in new models capable of generating high-quality images and creative artefacts, most of which seek to fully automate the process of creation. In stark contrast, creative professionals rely on iteration—to change their mind, to modify their sketches, and to re-imagine. For that reason, end-to-end generative approaches limit application to real-world design workflows. We present a novel human-AI drawing interface called Reframer… Mostrar más

Over the past few years, rapid developments in AI have resulted in new models capable of generating high-quality images and creative artefacts, most of which seek to fully automate the process of creation. In stark contrast, creative professionals rely on iteration—to change their mind, to modify their sketches, and to re-imagine. For that reason, end-to-end generative approaches limit application to real-world design workflows. We present a novel human-AI drawing interface called Reframer, along with a new survey instrument for evaluating co-creative systems. Based on a co-creative drawing model called the Collaborative, Interactive Context-Aware Design Agent (CICADA), Reframer uses CLIP-guided synthesis-by-optimisation to real-time synchronous drawing with AI. We present two versions of Reframer’s interface, one that prioritises emergence and system agency and the other control and agency. To begin exploring how these different interaction models might influence the experience, we also propose the Mixed-Initiative Creativity Index (MICSI). MICSI rates co-creative systems along experiential axes relevant to AI co-creation. We ister MICSI and a short qualitative interview to s who engaged with the Reframer variants on two distinct creative tasks. The results show overall broad efficacy of Reframer as a creativity tool, but MICSI also allows us to begin unpacking the complex interactions between learning effects, task type, visibility, control, and emergent behaviour. We conclude with a discussion of how these findings highlight challenges for future co-creative systems design. Mostrar menos

Recent advances in text-conditioned generative models have provided us with neural networks capable of creating images of astonishing quality, be they realistic, abstract, or even creative. These models have in common that (more or less explicitly) they all aim to produce a high-quality one-off output given certain conditions, and in that they are not well suited for a creative collaboration framework. Drawing on theories from cognitive science that model how professional designers and artists… Mostrar más

Recent advances in text-conditioned generative models have provided us with neural networks capable of creating images of astonishing quality, be they realistic, abstract, or even creative. These models have in common that (more or less explicitly) they all aim to produce a high-quality one-off output given certain conditions, and in that they are not well suited for a creative collaboration framework. Drawing on theories from cognitive science that model how professional designers and artists think, we argue how this setting differs from the former and introduce CICADA: a Collaborative, Interactive Context-Aware Drawing Agent. CICADA uses a vector-based synthesis-by-optimisation method to take a partial sketch (such as might be provided by a ) and develop it towards a goal by adding and/or sensibly modifying traces. Given that this topic has been scarcely explored, we also introduce a way to evaluate desired characteristics of a model in this context by means of proposing a diversity measure. CICADA is shown to produce sketches of quality comparable to a human 's, enhanced diversity and most importantly to be able to cope with change by continuing the sketch minding the 's contributions in a flexible manner. Mostrar menos

Recent advances in generative machine learning, particularly in the area of text-to-image synthesis, have
created huge potential for co-creative systems. It is non-trivial, however, to adapt algorithms intended to
generate images that match a given prompt to suit the task of effective collaboration with humans. This pa-
per presents initial experimentation towards developing an agent that can work cooperatively with a human designer in the task of drawing. We do so by utilizing… Mostrar más

Recent advances in generative machine learning, particularly in the area of text-to-image synthesis, have
created huge potential for co-creative systems. It is non-trivial, however, to adapt algorithms intended to
generate images that match a given prompt to suit the task of effective collaboration with humans. This pa-
per presents initial experimentation towards developing an agent that can work cooperatively with a human designer in the task of drawing. We do so by utilizing Contrastive Language Image Pretraining (CLIP) to guide the drawing’s semantic meaning on a drawing completion process, and fidelity to enforce geometric alignment (with what would be the ’s in-progress sketch). Preliminary results are presented as a proof of concept, attesting that drawing outputs are both diverse and identifiable as matching the provided prompt, which we interpret as steps towards co-creativity. Mostrar menos

Extreme Learning Machines (ELMs) have become a popular tool in the field of Artificial Intelligence due to their very high training speed and generalization capabilities. Another advantage is that they have a single hyper-parameter that must be tuned up: the number of hidden nodes. Most traditional approaches dictate that this parameter should be chosen smaller than the number of available training samples in order to avoid over-fitting. In fact, it has been proved that choosing the number of… Mostrar más

Extreme Learning Machines (ELMs) have become a popular tool in the field of Artificial Intelligence due to their very high training speed and generalization capabilities. Another advantage is that they have a single hyper-parameter that must be tuned up: the number of hidden nodes. Most traditional approaches dictate that this parameter should be chosen smaller than the number of available training samples in order to avoid over-fitting. In fact, it has been proved that choosing the number of hidden nodes equal to the number of training samples yields a perfect training classification with probability 1 (w.r.t. the random parameter initialization). In this article we argue that in spite of this, in some cases it may be beneficial to choose a much larger number of hidden nodes, depending on certain properties of the data. We explain why this happens and show some examples to illustrate how the model behaves. In addition, we present a pruning algorithm to cope with the additional computational burden associated to the enlarged ELM. Experimental results using electroencephalography (EEG) signals show an improvement in performance with respect to traditional ELM approaches, while diminishing the extra computing time associated to the use of large architectures. Mostrar menos

Generative models are undoubtedly a hot topic in Artificial Intelligence, among which the most common type is Generative Adversarial Networks (GANs). These architectures let one synthesise artificial datasets by implicitly modelling the underlying probability distribution of a real-world training dataset. With the introduction of Conditional GANs and their variants, these methods were extended to generating samples conditioned on ancillary information available for each sample within the… Mostrar más

Generative models are undoubtedly a hot topic in Artificial Intelligence, among which the most common type is Generative Adversarial Networks (GANs). These architectures let one synthesise artificial datasets by implicitly modelling the underlying probability distribution of a real-world training dataset. With the introduction of Conditional GANs and their variants, these methods were extended to generating samples conditioned on ancillary information available for each sample within the dataset. From a practical standpoint, however, one might desire to generate data conditioned on partial information. That is, only a subset of the ancillary conditioning variables might be of interest when synthesising data. In this work, we argue that standard Conditional GANs are not suitable for such a task and propose a new Adversarial Network architecture and training strategy to deal with the ensuing problems. Experiments illustrating the value of the proposed approach in digit and face image synthesis under partial conditioning information are presented, showing that the proposed method can effectively outperform the standard approach under these circumstances. Mostrar menos

When recorded in an enclosed room, a sound signal will most certainly get affected by reverberation. This not only undermines audio quality, but also poses a problem for many human-machine interaction technologies that use speech as their input. In this paper, a new blind, two-stage dereverberation approach based in a generalized β -divergence as a fidelity term over a non-negative representation is proposed. The first stage consists of learning the spectral structure of the signal solely from… Mostrar más

When recorded in an enclosed room, a sound signal will most certainly get affected by reverberation. This not only undermines audio quality, but also poses a problem for many human-machine interaction technologies that use speech as their input. In this paper, a new blind, two-stage dereverberation approach based in a generalized β -divergence as a fidelity term over a non-negative representation is proposed. The first stage consists of learning the spectral structure of the signal solely from the observed spectrogram, while the second stage is devoted to model reverberation. Both steps are taken by minimizing a cost function in which the aim is put either in constructing a dictionary or a good representation by changing the divergence involved. In addition, an approach for finding an optimal fidelity parameter for dictionary learning is proposed. An algorithm for implementing the proposed method is described and tested against state-of-the-art methods. Results show improvements for both artificial reverberation and real recordings. Mostrar menos

When a signal is recorded in an enclosed room, it typically gets affected by reverberation. This degradation represents a problem when dealing with audio signals, particularly in the field of speech signal processing, such as automatic speech recognition. Although there are some approaches to deal with this issue that are quite satisfactory under certain conditions, constructing a method that works well in a general context still poses a significant challenge. In this article, we propose a… Mostrar más

When a signal is recorded in an enclosed room, it typically gets affected by reverberation. This degradation represents a problem when dealing with audio signals, particularly in the field of speech signal processing, such as automatic speech recognition. Although there are some approaches to deal with this issue that are quite satisfactory under certain conditions, constructing a method that works well in a general context still poses a significant challenge. In this article, we propose a Bayesian approach based on convolutive nonnegative matrix factorization that uses prior distributions in order to impose certain characteristics over the time-frequency components of the restored signal and the reverberant components. An algorithm for implementing the method is described and tested. Comparisons of the results against those obtained with state-of-the-art methods are presented, showing significant improvement. Mostrar menos

The image inpainting problem consists of restoring an image from a (possibly noisy) observation, in which data from one or more regions are missing. Several inpainting models to perform this task have been developed, and although some of them perform reasonably well in certain types of images, quite a few issues are yet to be sorted out. For instance, if the image is expected to be smooth, the inpainting can be made with very good results by means of a Bayesian approach and a maximum a… Mostrar más

The image inpainting problem consists of restoring an image from a (possibly noisy) observation, in which data from one or more regions are missing. Several inpainting models to perform this task have been developed, and although some of them perform reasonably well in certain types of images, quite a few issues are yet to be sorted out. For instance, if the image is expected to be smooth, the inpainting can be made with very good results by means of a Bayesian approach and a maximum a posteriori computation. For non-smooth images, however, such an approach is far from being satisfactory. Even though the introduction of anisotropy by prior smooth gradient inpainting to the latter methodology is known to produce satisfactory results for slim missing regions, the quality of the restoration decays as the occluded regions widen. On the other hand, Total Variation (TV) inpainting models based on high order PDE diffusion equations can be used whenever edge restoration is a priority. More recently, the introduction of spatially variant conductivity coefficients on these models, such as in the case of Curvature-Driven Diffusion (CDD), has allowed inpainted images with well defined edges and enhanced object connectivity. The CDD approach, nonetheless, is not quite suitable wherever the image is smooth, as it tends to produce piecewise constant restorations.

In this work we present a two-step inpainting process. The first step consists of using a CDD inpainting to build a pilot image from which to infer a-priori structural information on the image gradient. The second step is inpainting the image by minimizing a mixed spatially variant anisotropic functional, whose weight and penalization directions are based upon the aforementioned pilot image. Results are presented along with comparison measures in order to illustrate the performance of this inpainting method. Mostrar menos

During the last two decades several generalizations of the traditional Tikhonov–Phillips regularization method for solving inverse ill-posed problems have been proposed. Many of these variants consist essentially of modifications on the penalizing term, which force certain features in the obtained regularized solution. If it is known that the regularity of the exact solution is inhomogeneous it is often desirable the use of mixed, spatially adaptive methods. These methods are also highly… Mostrar más

During the last two decades several generalizations of the traditional Tikhonov–Phillips regularization method for solving inverse ill-posed problems have been proposed. Many of these variants consist essentially of modifications on the penalizing term, which force certain features in the obtained regularized solution. If it is known that the regularity of the exact solution is inhomogeneous it is often desirable the use of mixed, spatially adaptive methods. These methods are also highly suitable when the preservation of edges is an important issue, since they allow for the inclusion of anisotropic penalizers for border detection. In this work we propose the use of a penalizer resulting from the convex spatially-adaptive combination of a classic penalizer and an anisotropic bounded variation seminorm. Results on existence and uniqueness of minimizers of the corresponding Tikhonov–Phillips functional are presented. Results on the stability of those minimizers with respect to perturbations in the data, in the regularization parameter and in the operator are also established. Applications to image restoration problems are shown. Mostrar menos