Amazon QuickSight is a fully managed Business Intelligence (BI) service for data visualization.
Continue reading “[AWS] Amazon QuickSight”Author Archives: Pyongwon Lee
[AWS Lab] VPC – Public Subnet
In this lab, we will learn how to create a public subnet and test it with an EC2 instance.
Overview
- VPC
- Create a VPC and public subnet
- EC2
- Create an EC2 instance inside the public subnet
- Setup the website in the EC2 instance
- Access the website via the public IP address of the EC2 instance
[AWS Lab] WAF – Rate-baed Rule
In this lab, we will learn how to block traffic through WAF.
- API Gateway – REST API
- Route: GET /greet
- Lambda – Function
- Return the greeting message
- Python
- WAF
- Set up the rate-based rule
- Link to the API
[Spark By Example] Read SQL Server
The following sample code (by C#) shows how to read data from the Microsoft SQL Server.
Continue reading “[Spark By Example] Read SQL Server”[AWS] Observability
One of the biggest challenges in developing and host applications in cloud is how to monitor and debugging issues. This post presents the basic concept of observability and AWS’s services.
Continue reading “[AWS] Observability”[Spark By Example] Schema
Spark can infer the data structure, but you can explicitly specify the data by providing the Schema to the DataFrame.
Continue reading “[Spark By Example] Schema”[Spark By Example] SparkSession
The following sample code (by Python and C#) shows how to use SparkSession.
SparkSession
- SparkSession is an entry point to your spark application since Spark version 2.
- SparkSession wraps all different contexts (SparkContext, SQLContext, HiveContext, …) to a single entry point.
- You can create as many SparkSessions as you want.
- In the Spark shell, such as PySpark shell, the SparkSession object (named as “spark”) is created for you.
- In the application, you need to create a SparkSession object.
[AWS Lab] Lambda – /tmp Space
In this lab, we will learn how to use local transient cache (/tmp) folder in a Lambda function.
- Overview
- S3
- Create a bucket
- Upload a configuration file
- Lambda – Function
- Retrieves the configuration file from S3 or /tmp
- Python, boto3