JPEGmini Server for AWS

JPEGmini Server for AWS

JPEGmini Server for AWS

GETTING STARTED

GETTING STARTED

Overview

In this tutorial, you will learn how to launch a JPEGmini Server running on the Amazon Web Services (AWS) cloud infrastructure in 5 simple steps. This tutorial covers all the steps required for creating and launching a JPEGmini server instance on AWS, and optimizing photos stored on your computer.

Deploy a JPEGmini Server and Optimize Your Photos in 5 Simple Steps:

Step 1. Sign up for an AWS account

Step 1. Sign up for an AWS account

If you already have an AWS account, skip to the next step. If you don't already have an AWS account, use the following procedure to create one.

To create an AWS account, go to http://aws.amazon.com, and click “Sign Up Now”. Follow the onscreen instructions to sign up. Part of the process involves receiving a phone call and entering a PIN using the phone keypad.

Note that when you create an AWS account, AWS automatically signs up the account for all AWS services, including Amazon EC2. However, you will only be charged for the services that you actually use.

Step 2. Launch your JPEGmini Server from the AWS Marketplace

Step 2. Launch your JPEGmini Server from the AWS Marketplace

The JPEGmini Server is available on the AWS Marketplace as a pre-configured Amazon Machine Image (AMI). To launch your server, go to the JPEGmini Server for AWS page. Make sure you are logged in with your AWS account.

Click on the “Continue” button to proceed to the launch page. Before launching the JPEGmini Server, you will need to select an EC2 Instance Type. In this tutorial we will use the Standard Small (m1.small) instance.

If this is your first time running an EC2 instance, you will need to create a Key Pair. The Key Pair is used to authenticate you when logging on to the EC2 server. If you already have a key-pair, select it in the key-pair section of the form, and skip to the next step.

If this is your first time running an EC2 instance, you will need to create a Key Pair. The Key Pair is used to authenticate you when logging on to the EC2 server. If you already have a key-pair, select it in the key-pair section of the form, and skip to the next step.

In a separate browser tab, open the EC2 Console and switch to the Key Pairs tab.

Click on “Create Key Pair”, set the name to “jpegmini-server-key”, and click “Create”. Your browser will download the private keyfile named jpegmini-server-key.pem, and a key-pair will show up in the ec2 web-console.

Now go back to the Marketplace browser tab, and refresh the page. Make sure your jpegmini-server-key is selected in the Key Pair section.

Launch the server by clicking “Accept Terms & Launch with 1-Click” It may take a few minutes for the server to launch - make sure not to leave the page until it launches.

If you get a dialog indicating that the subscription is taking longer than expected, don’t worry. You will probably receive a confirmation email within a few minutes. After receiving the confirmation email, go back to the JPEGmini Server for AWS page, and click on “Launch with 1-Click”. This time it should deploy faster, since you are already subscribed to the JPEGmini Server.

If all went well, you should see a page indicating that JPEGmini Server is now deploying on EC2.

Step 3. Connect to your running instance and run the JPEGmini Server

Step 3. Connect to your running instance and run the JPEGmini Server

Go the AWS console instances tab, select your instance, and copy the instance’s Public DNS to your clipboard.

Connect to your instance using an SSH (Secure Shell) terminal:

On Mac or Linux:

  1. Open a terminal window

  2. Secure your key, by entering the following command:Go the AWS console instances tab, select your instance, and copy the instance’s Public DNS to your clipboard.

chmod 600 ~/Downloads/jpegmini-server-key.pem

Note:
* The command above assumes your private key file is saved in the Downloads folder


3. Connect to the JPEGmini Server instance with the following command. When connecting for the first time, your computer will alert that the authenticity of the host cannot be established, and ask you if you want to continue connecting - just answer Yes.

ssh -i ~/Downloads/jpegmini-server-key.pem ec2-user@public-dns-name-here

Notes:
* The command above assumes your private key file is saved in the Downloads folder
* Use the public dns name previously copied from the AWS console


On Windows:

Windows doesn’t support SSH natively, so you should use a free program called PuTTY. Download PuTTY and PuTTYgen from the PuTTY Download Page

Once connected to your instance, you can use Linux shell commands to control the server. To run JPEGmini on the sample photos included with the server, execute the following command:

jpegmini -f=Demo1

The output should look like this:

============================================
Start jpegmini 3.9.17.0L (click any key to terminate)
============================================
INFO: Reading input file /home/ec2-user/Demo1/hipnshoot.jpg, file size=4814 KB
INFO: Writing output file /home/ec2-user/Demo1/hipnshoot_mini.jpg, file size=1304 KB
INFO: Recompression ratio: 3.69X (73% saving)


============================================
Finish jpegmini 3.9.17.0L

The above command ran the JPEGmini Server and compressed the photo in the folder “Demo1”. The console output shows that the file size of the photo was reduced by 73%.

Use the ls command to list the files in the folder “Demo1”:

ls -l Demo1

The output should look like this:

-rw-r--r-- 1 ec2-user ec2-user  4930482 May 8 19:54 hipnshoot.jpg
-rw-rw-r-- 1 ec2-user ec2-user  1335467 Jul 1 12:10 hipnshoot_mini.jpg
-rw-r--r-- 1 ec2-user ec2-user  149     May 8 19:54 license.txt

Note the smaller file size of hipnshoot_mini.jpg, which is the result of the optimization.

You can also connect to your instance via a browser, and access a dashboard that is used to provide information about your instance. To access the dashboard, point your browser to: http://your-public-dns-name

To quickly check that the server is online and active, browse to the following URL: http://your-public-dns-name/api/v1/heartbeat This method returns the current time and jpegmini version in json format.

Step 4. Upload photos to your server

Step 4. Upload photos to your server

In order to optimize your photos, you need to transfer them to the running instance of the JPEGmini sever on AWS. The best method to do so depends on the location of your photos (stored on s3, stored on ebs volumes, on premise, etc.). In this tutorial we will use the free FileZilla SFTP Client to copy files from your computer to the server or from the server to your computer.

Setting up FileZilla:

1. Download and install FileZilla to your computer.

2.Import the private key file by clicking on “Preferences” in the menu, switching to the SFTP tab (left), and clicking on “Add Keyfile”, and selecting your jepgmini-server-key.pem file

3. FileZilla will ask you to convert the .pem file into a supported file format. Click “Yes”, save it as jpegmini-server-key.ppk and click ok

4. At the top of the window, just below the toolbar, enter the following hostname, username, and port and click “Quickconnect”:
hostname: your public dns name copied from AWS console
username: ec2-user
Port: 22
Password: [Leave this field empty]

After clicking “Quickconnect”, FileZilla will show a dialog indicating the host could not be authenticated. Click “OK”.

Once connected you will see the server’s filesystem on the right side of the window, and your local computer's file system on the left side of the window. You can copy files from one side to another by dragging them.

In the middle-right window, right click on the ec2-user directory, select “Create directory” and create a new directory called photos. Click on photos to select it, and note the bottom right window indicating it is empty.

Copy over a couple of JPEG photos from your computer to the photos folder by dragging them onto the photos folder. On Mac computers, you can drag photos directly from within the Finder or iPhoto. On Windows, you can drag them directly from within the File Explorer. Wait for the photos to upload, and make sure you can see them inside the photos directory on the server’s filesystem (right side).

Step 5. Optimize your photos

Step 5. Optimize your photos

You are now ready to optimize the photos you have uploaded in Step 4. Switch back to the Terminal window used in Step 3, use the ls command to list the files and folders, and make sure you see the photos directory you created in Step 4.

Run JPEGmini Server with the following command to optimize all JPEG files in the photos directory:

jpegmini -f=photos

Finally, use FileZilla to copy the optimized jpeg photos back to your computer as follows:

  1. Return to the FileZilla application

  2. Press F5 to refresh the file listing

  3. On the remote server (right side) select the photos folder (/home/ec2-user/photos)

  4. On the local server (left side) select your Dekstop (/Users/username/Desktop)

  5. Right click on the files with _mini suffix in the right window, and select Download. After downloading is completed, the optimized photos will be on your desktop.

Congratulations!
If you made it here, you have successfully optimized your photos.

OPERATING INSTRUCTIONS

OPERATING INSTRUCTIONS

General

The JPEGmini Server application takes a JPEG file as input, and outputs a smaller JPEG file with the extension “_mini.jpg”. The application can also be used on a folder of photos. When processing is finished, the application reports the sizes of the original and re-compressed photos, and the reduction factor achieved.

Supported Formats

The JPEGmini Server supports input files in standard JPEG format. Supported resolutions are from 32x32 pixels up to 60 Megapixels.

Typical Usage

Typical usage only requires specifying the input file name:

jpegmini –f=test.jpg

The output in this case will be the file test_mini.jpg. The application can also be executed on a complete folder, for example:

jpegmini –f=User/TestImages

In this case, all files in the folder User/TestImages will be processed by JPEGmini.

Command Line Options

Command Line Options

The JPEGmini application supports the following command line options:

-help: Displays usage instructions. Any other flags used in combination with –help will be ignored.

-f: Specifies the input JPEG file name or folder name which will be processed by JPEGmini.


-f=<file name>: Specifies an input JPEG file name (JPEGmini will process only the specified input JPEG file).


-f=<folder path>: Specifies the input JPEG folder name (JPEGmini will process all the JPEG files in the specified folder).


-r: Specifies whether to recursively process all subfolders in a specified folder (relevant only when using the -f=<folder path> command line option to specify an input folder name).


-r=1: Recursively process all subfolders in the specified folder [default].


-r=0: Process the specified folder only, without its subfolders.


-o: Specifies the file name or folder name for the output images. When this flag is not used, by default the output file names are the same as the input file names with the postfix “_mini”, and they are placed in the same folder as the input files.


-o=<file name>: Specifies the name for the output JPEG file (relevant only when using the -f=<file name> command line option to specify an input file name).


-o=<folder path>: Specifies the folder in which the output JPEG files will be placed.


-logfile=<file name>: Specifies the destination file for writing the JPEGmini log information printouts (See the “Application Printouts” section below). If this flag is not specified, by default JPEGmini sends the log information printouts to the console.


-lvl: Specifies the level of log information that JPEGmini will print to the console or to the specified log file (in case the -logfile=<file name> command line option was used).


-lvl=0: No logs.


-lvl=1: Basic log level [default].


-lvl=2: Advanced log level


-csvfile=<file name>

The processing summary file is written in addition to the log information output.

The processing summary file includes 5 fields for each processed file:

filein: Name of input file


fileout: Name of output file


sizein: Size of input file in bytes


sizeout: Size of output file in bytes


err: Error code in HEX (0x0 for success)


-rsz: Resize the image before recompression.


-rsz=<x>: Resize the image to x% of the original width and height, preserving the aspect ratio. The value of x is from 1 to 99.


-rsz=width x height: Resize the image to width x height pixels, without preserving the aspect ratio. width and height must be larger than 31, and smaller than the original image’s width and height.


-qual: Specifies the quality of the output JPEG image.


-qual=0: Best Quality [default]. Produces an output image that is perceptually identical to the input image.


-qual=1: High Quality. Produces an image with a higher compression ratio than “Best Quality”, which still has a high quality, and provides a good tradeoff between quality and file size.


-qual=2: Medium Quality. Provides the highest compression level (smallest file size) at the expense of image quality.


-shc: Specifies whether to skip highly compressed images. When this flag is turned on, the application analyzes the input image to determine if it has already undergone heavy compression. If this is the case, the input file will be copied to the output without applying JPEGmini processing. The purpose of this option is to prevent the overhead of processing input files for which JPEGmini cannot provide a significant additional compression gain.


-shc=0: Don’t skip highly compressed images.


-shc=1: Skip highly compressed images [default].


-rmt: Specifies whether to remove metadata from the JPEG image. Removing metadata deletes the auxiliary information associated with the JPEG image, such as date, location, camera model, etc. Removing metadata can further reduce the output file size, especially for small input files.


-rmt=0: Don’t remove metadata from the JPEG file [default]

Application Printouts

Application Printouts

During its execution, the application prints out the following information:

  • Application version

  • Name and size of input JPEG file

  • Name and size of output JPEG file

  • Recompression ratio: The ratio between the input and output file sizes, and the difference between them in percent.


Below is an example of a typical printout for a file processed by JPEGmini.

Reading input file test\image1.jpg, file size=1498 KB
Writing output file test\image1_mini.jpg, file size=376 KB
Recompression ratio: 3.98X (74% saving)

When executing JPEGmini on a folder (using the -f=<folder path> command line option), the above information is displayed for each processed file. After completing the processing of all files in the folder, the application prints out the following summary information:

  • Total number of processed files

  • Total size of input files

  • Total size of output files

  • Recompression ratio: The ratio between the total size of the input files and the total size of the output files, and the difference between them in percent.


Below is an example of a typical folder summary report displayed by JPEGmini.

------------------------------------------
Summary report for folder test [including subfolders]:
Total number of files: 23
Total size of input files: 79356 KB
Total size of output files: 33370 KB
Recompression ratio: 2.38X (57% saving)

Note that this summary information refers to all the files which were processed by JPEGmini during the current execution, including files in subfolders (if the –r=1 option was used).

Terminating the application

Pressing any key during the execution of JPEGmini will cause the following dialog to be displayed:

Do you want to terminate jpegmini? [Y,N]

Pressing “Y” at this stage will cause the application to terminate.
Pressing “N” at this stage will cancel the termination, and execution of JPEGmini will continue.

USAGE NOTES

USAGE NOTES

The recompression ratios achieved depend on the image resolution, quality and content, and therefore are different for each image. However, the perceptual quality of the resulting image will always be the same as the original, when -qual=0 (Best quality) is used.

JPEGmini generates the highest recompression ratios for first-generation JPEG images that are created by digital cameras (stand-alone devices or embedded in cellphones). Using JPEGmini on images that have been edited, re-sized or processed by image editing software may cause lower recompression ratios.

If JPEGmini skips input images without processing them (for example if their resolution is not supported, or if they are highly compressed and the –shc flag is turned on), the input images are copied to the output with the extension “_copy.jpg”.

JPEGmini only supports JPEG images as input. Non-JPEG images can be converted to JPEG and then optimized by JPEGmini. In this case, it is recommended to convert the images to JPEG at a JPEG quality (QF) of 100.

USING JPEGMINI FOR AWS WITH S3

USING JPEGMINI FOR AWS WITH S3

Move photos back and forth between Amazon S3 and your JPEGmini for AWS instance

Overview

In this tutorial, you will learn how to copy photos from Amazon S3 to your JPEGmini Server running on AWS, optimize them, and copy the optimized photos back to S3. “Amazon S3 is the storage for the internet”, therefore a simple method of processing photos stored on s3 is extremely valuable. For more details about S3, see the S3 home page.

Prerequisites:

A running instance of the JPEGmini Photos Server

Step 1. Setup access credentials for your JPEGmini instance

Before accessing S3 from your running instance on AWS, you will need to setup the appropriate access credentials. AWS supports user-based access tokens, groups and roles. In this tutorial we will create a dedicated user for the JPEGmini Server instance, and grant that user full access to S3.


Note that on a production environment, you would probably want more restrictive access for increased security. For additional information on access management, see the IAM documentation.

To create the user, follow these steps:

1. Login to your AWS console and switch to the IAM service:
https://console.aws.amazon.com/iam/home

1. Select the Users tab (on the left), and click on “Create New User”

3. In the “Create User” dialog, enter just a single name: “jpegmini”, and click on “Create”

4. Click on “Download Credentials” to download a file containing the access id and secret access key for this new user.

5. Close the dialog

To grant the jpegmini user full access to S3, follow these steps:

1. From the IAM console, select the jpegmini user

2. Open the Permissions tab (bottom section of the screen), and click “Attach User Policy”

3. Select “Amazon S3 Full Access”, click “Select”, and then click “Apply Policy”

Step 2. Create an S3 bucket, and place some photos in it

There are various tools to manage S3, create buckets, and transfer files between S3 and your computer (e.g. AWS Console, S3browser, etc). In this tutorial we use the AWS console to create a bucket and upload some files to it. For additional information about S3, see the Amazon S3 documentation.

To create an S3 bucket for your photos, follow these steps:

1. Login to the S3 console

2. Click on the “Create Bucket” button on the top left

3. In the dialog, set the bucket name to any unique name and click on “Create”

Note the bucket names must be globally unique (i.e. two users cannot have the same bucket names). Repeat the three previous steps to create a second bucket with a different name to hold the optimized photos.

To upload photos to your S3 bucket, follow these steps in the AWS console:

1. Double click on the bucket name to enter the bucket

2. Click on the blue “Upload” button to open the upload dialog

3. Click on Add Files, and select a few jpeg photos from your computer

4. To make the files publicly accessible for download, you can click on “Set Details”, followed by “Set Permissions”, and check “Make everything public”

5. Click on “Start Upload” to upload your photos

By making the files public, you can view each photo directly from your browser. Get the file URL for one of the photos by right-clicking on it and selecting “Properties” from the context menu. The link to the file is shown on the right.

Step 3. Copy the original photos from your S3 bucket to your JPEGmini Instance

In this tutorial we use s3cmd to copy files between S3 and the JPEGmini Server instance. s3cmd is pre-installed on the JPEGmini Server instance. It is a simple command line tool that can easily be automated (scripted) to manage all S3 file transfers.

Before using s3cmd for the first time, you will need to configure it with the credentials generated in the first step. Login to your JPEGmini instance, and run the following command:

s3cmd --configure

Copy the Access Key and Secret Key from the credentials.csv file which was downloaded in the previous step. Use the default answer for all other items, and when prompted to “Save Settings”, answer “Yes”

To list your S3 buckets and files use the following commands:

s3cmd ls
s3cmd ls s3://your_back_name

To copy files or folders from S3 to your instance, use the get command:

s3cmd get s3://your_back_name/your-photo-name
s3cmd get --recursive s3://your-bucket-name

Copy the original photos from your S3 bucket into a folder called ~/my_photos on your running instance:

cd
mkdir my_photos
cd my_photos
s3cmd get --recursive s3://your-bucket-name/
ls -l

You should now see a list of your photo files, which are now located on the server.

Step 4. Optimize your photos

Change back to your home folder, and run JPEGmini as follows (depending on the number of photos this may take some time):

cd
mkdir my_optimized_photos
jpegmini -f=my_photos -o=my_optimized_photos
ls -l my_optimized_photos

You should now see a list of files (your optimized photos). Note the files should have a _mini suffix in their filename, and their sizes should be smaller than the original files.

Step 5. Copy the optimized photos back to your S3 bucket

The last step is to upload the optimized photos back to S3. Use the following command to run s3cmd and upload the optimized photos:

s3cmd put -P ~/my_optimized_photos/* s3://your-2nd-bucket-name

The -P flag is used to make the files publicly available for download. Go back to the S3 Console to verify that the optimized photos have been placed in the bucket.

JPEGMINI FOR AWS REST API

JPEGMINI FOR AWS REST API

JPEGmini REST API provides methods to optimize jpeg images.

Optimize method

Method: POST
URL: http://<PUBLIC_HOST>/api/v1/optimize
Body: The data of the source jpeg photo

Optional Query Parameters:

Optional Query Parameters:

  • scale - scale factor between 0 and 1

  • width - target width (in pixels)

  • height - target height (in pixels)


Usage notes

  • If no parameters are provided, the image size will be maintained (scale=1.0)

  • You cannot pass both scale and (width or height)

  • If only width, or only height are provided, the original aspect ration is maintained


CURL example

curl -X POST -H 'Content-Type: image/jpeg' \ --data-binary @/tmp/input.jpg --output /tmp/output.jpg \ http:///api/v1/optimize?width=800

-X POST - specify http POST method

  • -H 'Content-Type: image/jpeg' - specify body content is a jpeg

  • --data-binary @/tmp/input.jpg - send /tmp/input.jpg as the data of the request

  • --output /tmp/output.jpg - store response in /tmp/output.jpg (instead of stdout)

  • http:///api/v1/optimize - the request url

  • ?width=800 - resize the image to target width of 800 pixels

Python example

import requests
jpegmini_server = ""
with open('/tmp/input.jpg') as src:
    with open('/tmp/output.jpg', 'wb') as dst:
        response = requests.post(
             url="http://%s/api/v1/optimize" % jpegmini_server, data=src)
        dst.write(response.content)

C# example

static public void OptimizePhoto(String inputFile, String outputFile) {
  String host = "";

  //Setup the web POST request
  HttpWebRequest request = (HttpWebRequest)WebRequest.Create(
  String.Format("http://{0}/api/v1/optimize", host));
  request.Method = "POST";

  //Upload the file
  FileStream fs = new FileStream(inputFile, FileMode.Open);
  fs.CopyTo(httpWebRequest.GetRequestStream());

  //Get the server response, and write it a file
  WebResponse response = httpWebRequest.GetResponse();
  FileStream of = new FileStream(outputFile, FileMode.Create);
  response.GetResponseStream().CopyTo(of

Proxy method

Method: GET
URL: http:///api/v1/proxy
Query parameters:

  • url - url to the source image

Usage notes

  • This method caches results in ElasticCache (memcache / redis) if configured

  • The returned http response will include the following headers:

  • x-jpegmini-src - size of original image

  • x-jpegmini-mini - size of optimized image

  • x-jpegmini-reduced - number of bytes saved

  • x-jpegmini-reduced_p - percent of bytes saved

  • x-jpegmini-hit - true if result is returned from cache

  • x-jpegmini-error - jpegmini error (in case of error)


CURL example

curl -v -X GET --output /tmp/output.jpg \ http:///api/v1/proxy? \ url=http://awsmarketplace.jpegmini.com/PhotKing.jpg
  • -v - verbose (in order to see headers)

  • -X GET - specify http GET method

  • --output /tmp/output.jpg - store response in /tmp/output.jpg (instead of stdout)

  • http:///api/v1/proxy - the request url

  • ?url=http://awsmarketplace.jpegmini.com/PhotKing.jpg - the url of the image to optimize

Python example

import requests

jpegmini_server = ""
src_img = "http://awsmarketplace.jpegmini.com/PhotKing.jpg"
with open('/tmp/output.jpg', 'wb') as dst:
    response = requests.get(
        url="http://%s/api/v1/proxy?url=%s" % (
        jpegmini_server, src_img))
    print response.headers
    dst.write(response.content)

C# example

static public void OptimizePhoto(String outputFile) {
    String host = "";
    String srcImage = "http://awsmarketplace.jpegmini.com/PhotKing.jpg";

    //Setup the web POST request
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(
    String.Format("http://{0}/api/v1/proxy?url={1}", host, url));
    request.Method = "GET";

    //Get the server response, and write it a file
    WebResponse response = httpWebRequest.GetResponse();
    FileStream of = new FileStream(outputFile, FileMode.Create);
    response.GetResponseStream().CopyTo(of

EXIT CODES

The table below provides information about the exit codes of the JPEGmini Server.



PERFORMANCE BENCHMARK

The benchmarks below provide typical performance figures for the JPEGmini Server.

Camera Photos

Web Photos

Number of Photos

30

30

30

1000

Resolution

8 Megapixels (3264 x 2448)

8 Megapixels (3264 x 2448)

8 Megapixels (3264 x 2448)

from 80x80 to 140x140 pixels

Average file size reduction

3.02x (67% saving)

1.34x (26% saving)

1.34x (26% saving)

1.34x (26% saving)

Total processing time

36 seconds

3 seconds

The benchmarks were measured by running one process of the JPEGmini Photo Server on the following configuration:

Instance Type:Amazon c3.large instance
OS:Ubuntu 14.04 Server
Storage: Magnetic disk
* Performance may vary depending on system configuration and type of images.


The benchmarks were measured by running one process of the JPEGmini Photo Server on the following configuration:

Instance Type:Amazon c3.large instance
OS:Ubuntu 14.04 Server
Storage: Magnetic disk
* Performance may vary depending on system configuration and type of images.

Camera Photos

Web Photos

Number of photos

30

1000

Resolution

8 Megapixels (3264 x 2448)

from 80x80 to 140x140 pixels

Average file size reduction

3.02x (67% saving)

1.34x (26% saving)

Total processing time

36 seconds

3 seconds