Sao Paulo, San Pablo, Brasil
3 mil seguidores Más de 500 os

Unirse para ver el perfil

Acerca de

I’m a senior data engineer with a strong ion for data. With over 8 years of…

Actividad

Unirse para ver toda la actividad

Experiencia y educación

  • Itaú Unibanco

Mira la experiencia completa de Ericles

Mira su cargo, antigüedad y más

o

Al hacer clic en «Continuar» para unirte o iniciar sesión, aceptas las Condiciones de uso, la Política de privacidad y la Política de cookies de LinkedIn.

Licencias y certificaciones

Cursos

  • Técnico de Informática

    2 anos

Proyectos

  • Move to Cloud

    Decommission of the Cloudera clusters due to high costs and performance loss in the databases on the bank's on-premise servers. To solve this problem, it was necessary to migrate the entire data lake and legacy applications to Azure's cloud. This challenge included legacy bases with over 3,000 daily jobs feeding them from legacy sources such as the mainframe, SQL Server, Oracle Exadata Database Machine, SaaS, MySQL, and PostgreSQL.

    My team and I built capabilities to serve as a data…

    Decommission of the Cloudera clusters due to high costs and performance loss in the databases on the bank's on-premise servers. To solve this problem, it was necessary to migrate the entire data lake and legacy applications to Azure's cloud. This challenge included legacy bases with over 3,000 daily jobs feeding them from legacy sources such as the mainframe, SQL Server, Oracle Exadata Database Machine, SaaS, MySQL, and PostgreSQL.

    My team and I built capabilities to serve as a data platform. This platform was designed to the bank's teams in migrating their data and applications (jobs) to the cloud environment. The architecture devised for this migration facilitated the work of the bank's teams, leveraging a cloud-native stack from Azure, including Azure Data Lake Storage Gen2, Data Factory, Databricks, Key Vault, and service principal names (SPNs).

    We developed a Python library (PySpark 3) for ETL processes, ensuring governance, quality, performance, access control, and compliance with LGPD, Brazil's General Data Protection Law. Having managed this library from 2021, we've migrated 80% of the data lake's data and 40% of the applications. This migration is expected to reduce costs by about 30% in the coming years.

  • New infrastructure

    -

    I worked on the legacy infrastructure replacement project, besides being expensive, it did not allow scalability and data traffic in real time. There were also serious security and unavailability problems.
    We have implemented a new topology capable of working with data received from RaspBerrys (IOT) in real time on the end customer Dashboard.

    Legacy infrastructure:
    AWS: 35 EC2, 2 MySQL RDS, 1 RDS SQL Server, 16 SQS Queues, 17 Lambda Functions, 57 S3 Buckets, 2 ElastiCache (REDIS)…

    I worked on the legacy infrastructure replacement project, besides being expensive, it did not allow scalability and data traffic in real time. There were also serious security and unavailability problems.
    We have implemented a new topology capable of working with data received from RaspBerrys (IOT) in real time on the end customer Dashboard.

    Legacy infrastructure:
    AWS: 35 EC2, 2 MySQL RDS, 1 RDS SQL Server, 16 SQS Queues, 17 Lambda Functions, 57 S3 Buckets, 2 ElastiCache (REDIS), 10 API Gateway, 2 Kinesis, 1 EFS, 7 Elastic IPs, 18 Load Balancers, 54 Security Groups, 18 EBS Volumes, 1 Elasticsearch;
    PEERONE: 4 Physical machines in Texas;
    HEROKU: 2 Memcache, CloudAQMP Integrated 3 Queues, 5 Instances;
    GOOGLE CLOUD: 1 BigQuery.

    New infrastructure:
    AWS: 14 Buckets, 3 ECS Cluster, Farget: 20 Services, EC2: 7 Services, 1 Elastic IPs, 4 EC2, 14 EBS Volumes, 3 Load Balancers, 12 Security Groups, 13 Lambda Functions, 2 API Gateway, IoT, 1 RDS Aurora MySql, Route53 9 Routes, CloudFront 10 Distribution, 1 ACM Certificate, 1 EFS, Redis as Cache, InfluxDB TimeSeries running on an EC2;
    CLOUDAQMP: 1 Instance, 17 Queues;

    The main focus of this new topology was to economy cloud resources (to lower the cost) and at the same time solve the problems identified in the legacy infrastructure. I participated in the rewriting of applications and APIs before made in Java and PHP, for the languages: Python, NodeJS and ReactJS, with automated deployment using Docker and CI / CD.

    Ver proyecto

Idiomas

  • Português

    Competencia profesional completa

  • Inglês

    Competencia básica profesional

Más actividad de Ericles

Ver el perfil completo de Ericles

  • Descubrir a quién conocéis en común
  • Conseguir una presentación
  • ar con Ericles directamente
Unirse para ver el perfil completo

Perfiles similares

Añade nuevas aptitudes con estos cursos

Ver todos los cursos