Mateus Ferreira

he works as a data engineer responsible for ing various teams across Magazine Luiza. Their job involves maintaining and creating pipelines for specific business areas to ensure that all processes are executed efficiently and safely.

Additionally, he work closely with the governance team, constantly seeking alternatives to reduce pipeline costs and optimize existing cloud infrastructure. To do this, he use various important tools such as Kubernetes, Dataflow, Composer, Storage… Mostrar más

he works as a data engineer responsible for ing various teams across Magazine Luiza. Their job involves maintaining and creating pipelines for specific business areas to ensure that all processes are executed efficiently and safely.

Additionally, he work closely with the governance team, constantly seeking alternatives to reduce pipeline costs and optimize existing cloud infrastructure. To do this, he use various important tools such as Kubernetes, Dataflow, Composer, Storage, BigQuery, Looker Studio, Pyspark, and SQL.

In general, him work requires a lot of attention to detail and advanced technical skills in cloud environments. It is a dynamic and challenging job, but also very rewarding as they directly see the impact of their work on the company's process efficiency.

In addition to their daily responsibilities of maintaining and creating data pipelines, him also responsible for disseminating knowledge to other teams. As a data engineer, they must ensure that all team have a proper understanding of the processes and tools used, allowing them to have autonomy in creating their own data pipelines that meet specific business rules.

This responsibility includes conducting regular training, workshops, and mentoring sessions where the speaker shares their knowledge and experience in data engineering with other teams in the company. Him also provide technical and guidance to these teams, helping them find solutions to challenges faced during the pipeline creation process.

The dissemination of knowledge is an important part of him job, as it helps to increase the overall efficiency of the company and empowers teams to create their own data processes to meet specific business needs. Mostrar menos

Main roles:
- He acted as one of the main of the data engineering team at Johnson & Johnson, where he helped and ed the team during development.
- He was responsible for guiding other data engineers in their careers, acting as a career mentor.
- As part of him work, he also led the creation and migration of data pipelines, ensuring that data processes were working properly.
- He also worked on the application and creation of data quality flows, ensuring that data was… Mostrar más

Main roles:
- He acted as one of the main of the data engineering team at Johnson & Johnson, where he helped and ed the team during development.
- He was responsible for guiding other data engineers in their careers, acting as a career mentor.
- As part of him work, he also led the creation and migration of data pipelines, ensuring that data processes were working properly.
- He also worked on the application and creation of data quality flows, ensuring that data was accurate and of high quality for use in analysis and decision-making.

Environment:
- The project was developed in an Amazon Web Services cloud environment, which offered flexibility and scalability for our computing needs.

Technologies:
- Pyspark
- Glue
- DynamoDB
- Step Functions
- Lambda
- S3
- API Gateway
- Event Bridge
- Secret Manager
- Delta Lake
- Denodo Mostrar menos

He worked at Total Express as a data analyst, creating Dashboard suggestion for data presentation to business areas and utilization of tools for monitoring the data platform

Main actions:
- Analyze business areas' needs and define the key indicators to be presented on the dashboard
- Develop dashboards using Data Studio tool
- Perform tests and validate with end-s
- Configure and use the monitoring tools available on the platform
- Perform analyses to identify possible… Mostrar más

He worked at Total Express as a data analyst, creating Dashboard suggestion for data presentation to business areas and utilization of tools for monitoring the data platform

Main actions:
- Analyze business areas' needs and define the key indicators to be presented on the dashboard
- Develop dashboards using Data Studio tool
- Perform tests and validate with end-s
- Configure and use the monitoring tools available on the platform
- Perform analyses to identify possible problems and propose solutions
- Create reports with the key performance indicators of the platform

Environment:
- Amazon Web Service

Technologies:
- Data Studio
- CloudWatch
- Grafana Mostrar menos

Main activities:
- As a data engineer, he acted as a technical reference for the team, providing guidance and solutions for data-related challenges.
- One of him main projects was to remodel the company's data environment, giving autonomy for the use of airflow as an orchestrator, which resulted in a significant improvement in the efficiency and quality of ETL (extraction, transformation, and loading) tasks.
- Another project he developed was creating a machine learning model to… Mostrar más

Main activities:
- As a data engineer, he acted as a technical reference for the team, providing guidance and solutions for data-related challenges.
- One of him main projects was to remodel the company's data environment, giving autonomy for the use of airflow as an orchestrator, which resulted in a significant improvement in the efficiency and quality of ETL (extraction, transformation, and loading) tasks.
- Another project he developed was creating a machine learning model to classify comments on the company's app. This helped the team identify issues more quickly and improve the experience.

Environment:
- The company used the Google Cloud Platform cloud platform to host its data and applications.

Technologies:
- Pyspark
- Airflow
- Google Data Studio
- Google Cloud Storage
- Bigquery
- Bash Mostrar menos

Main activities:
- Creation of data collection and transformation pipelines, he were responsible for creating data pipelines using technologies such as Pyspark and Scala, ensuring the quality of collected and transformed data.
- Participation in the development of data quality flow, he actively participated in the development of the data quality flow, with the objective of ensuring the integrity and quality of the data used by the company.
- He had the opportunity to work with Cloud… Mostrar más

Main activities:
- Creation of data collection and transformation pipelines, he were responsible for creating data pipelines using technologies such as Pyspark and Scala, ensuring the quality of collected and transformed data.
- Participation in the development of data quality flow, he actively participated in the development of the data quality flow, with the objective of ensuring the integrity and quality of the data used by the company.
- He had the opportunity to work with Cloud for the first time, using the Microsoft Azure platform.

Environment:
- The company used the Microsoft Azure Cloud platform to host its data and applications.

Technologies:
- Pyspark
- Scala
- Databricks
- Azure Datafactory
- Azure devops
- Delta Lake
- Power BI Mostrar menos

They worked on a project called Rorac, where they performed several important technical functions, such as being the technical reference for the team, creating pipelines, and applying business rules.

The work environment was on-premise, with a Hadoop ecosystem. Several technologies were used to implement the project, such as Hadoop, Hive, Impala, Spark, HDFS, Control-M, Pyspark, and Shell.

It was a challenging but rewarding project for them, as they were able to apply their… Mostrar más

They worked on a project called Rorac, where they performed several important technical functions, such as being the technical reference for the team, creating pipelines, and applying business rules.

The work environment was on-premise, with a Hadoop ecosystem. Several technologies were used to implement the project, such as Hadoop, Hive, Impala, Spark, HDFS, Control-M, Pyspark, and Shell.

It was a challenging but rewarding project for them, as they were able to apply their technical knowledge and contribute to the project's success. Mostrar menos

In their first project as a data engineer, they worked on creating pipelines for data migration from an on-premise Hadoop ecosystem to a new environment. Some of their main roles included applying the necessary business rules for migration, ensuring data quality and information integrity.

To carry out the project, they used various technologies such as Hadoop, Hive, Impala, Spark, HDFS, Control-M, Pyspark, and Shell. Additionally, they worked closely with the development team to ensure… Mostrar más

In their first project as a data engineer, they worked on creating pipelines for data migration from an on-premise Hadoop ecosystem to a new environment. Some of their main roles included applying the necessary business rules for migration, ensuring data quality and information integrity.

To carry out the project, they used various technologies such as Hadoop, Hive, Impala, Spark, HDFS, Control-M, Pyspark, and Shell. Additionally, they worked closely with the development team to ensure that data was transferred securely and efficiently.

In summary, their contribution was crucial to ensuring that the data migration was successful, and that the transferred data was of high quality and reliable for use by end-s. Mostrar menos