Archive

Posts Tagged ‘Email’

Episode 6: Handling incoming Email in your application

October 31, 2009 3 comments

Welcome to Episode 6 of this series. In this episode, we will learn how your Google App Engine Java (GAEJ) Application can receive incoming email.

In an earlier episode of this series, we have covered how to use the Email Service of GAEJ to send out emails from your application. At that point in time, we were using version 1.2.5 of the AppEngine SDK. That version did not provide support for handling incoming email. Since then a newer version of the AppEngine SDK for Java 1.2.6 has been released. And one of the nice features of this new release is support for incoming email in your application. What it means is that anyone can send an email to your GAEJ hosted application, it can receive an email and then you can process and perform some business logic with that as required.

Prerequisites

Before we move on, it is important that you have a working setup of the environment and are comfortable with developing a new project using the Google Eclipse plugin. If not, please go through earlier episodes that contained detailed setup information along with a few development episodes like using Email Service, using the URL Fetch service, etc.

The most important prerequisite is to make sure that you have upgraded your Eclipse environment to the latest AppEngine SDK i.e. 1.2.6. Please go through the following : Episode 5: Upgrading to Google App Engine 1.2.6

A quick check: To make sure that you  have version 1.2.6 of the SDK installed, do the following:

1. In your Eclipse IDE, go to Window –> Preferences.
2. Navigate to Google –> App Engine
3. You should see version 1.2.6 of the SDK installed. And make sure that it is the default one by selecting it. By selecting it, it will be added to the Build Path of your Google Web Application projects. And we need the latest SDK since it will have support for the Incoming Email feature.

Receiving Email Feature

App Engine now supports incoming email in your applications. Read the official documentation here. Your applications can now receive email and you can parse out the email and determine any business logic that needs to be processed. This opens up a whole new range of applications where you can fulfill requests and send information from your application by allowing users to simply send an email from their favourite email client. Think of it as a Instant Message itself that your application can receive and react to it. We had seen in an earlier Episode how through XMPP Support, we can write our own Agent that can receive XMPP messages directly and respond to them. Now with version 1.2.6 of the SDK, the same functionality has got extended to email too. And the best part about it is the consistency with which Google has implemented it.

The steps to follow to receive an email is identical to the ones required to receive XMPP messages:

1. Configure your application to receive incoming email by configuring the Mail Service

2. Write and configure a servlet to receive email

3. Once the application is deployed, anyone can send an email to SomeID@YourApplicationId.appspotmail.com. SomeID is any id like test, admin, support,etc. It is just an id. And YourApplicationId is the application id of your hosted Google App Engine application.

Let us look at each of the above points in detail now. But before we begin, create a New Google Web Application Project (If you wish you can continue to use an existing project to add the incoming Email support, which is what I have done personally, but the choice is yours) . Follow these steps to create a new project:

1. Either click on File –> New –> Other or press Ctrl-N to create a new project. Select Google and then Web Application project. Alternately you could also click on the New Web Application Project Toolbar icon as part of the Google Eclipse plugin.
2. In the New Web Application Project dialog, deselect the Use Google Web Toolkit and give a name to your project. I have named mine GAEJExperiments. I suggest you go with the same name so that things are consistent with the rest of the article, but I leave that to you.
3. Click on Finish

This will generate the project and also create a sample Hello World Servlet for you. But we will be writing our own Servlet.

Configuring the incoming Email Service

This is straightforward and all you need to do is add the following element to the appengine-web.xml file. The appengine-web.xml file as you know is specific to the Google Java Web Application project and is used for configuring certain services. You need to configure the Incoming Email Service so that your application is enabled to receive it, being one of them. It is found in the war\WEB-INF folder of your Web Application Project. The XML fragment to add at the end but before the </appengine-web-app> element

<inbound-services>
<service>mail</service>
</inbound-services>

Configure and code a Java Servlet that will receive the incoming Message

All Email messages to your application are delivered via POST to following URL path in your application: /_ah/mail/ as per the Google AppEngine documentation. So you will need to configure the servlet like the following snippet in the web.xml file, present in the war\WEB-INF folder of your Web Application Project.

We need to add the <servlet/> and <servlet-mapping/> entry to the web.xml file. This file is present in the WEB-INF folder of the project. The necessary fragment to be added to your web.xml file are shown below. Please note that you can use your own namespace and servlet class. Just modify it accordingly if you do so.

 <servlet>
 <servlet-name>emailhandler</servlet-name>
 <servlet-class>com.gaejexperiments.email.GAEJReceiveEmailServlet</servlet-class>
 </servlet>
 <servlet-mapping>
 <servlet-name>emailhandler</servlet-name>
 <url-pattern>/_ah/mail/*</url-pattern>
 </servlet-mapping>

<security-constraint>
 <web-resource-collection>
 <url-pattern>/_ah/mail/*</url-pattern>
 </web-resource-collection>
 <auth-constraint>
 <role-name>admin</role-name>
 </auth-constraint>
</security-constraint>

 

In the above snippet, you will find the fixed URL path /_ah/mail/* configured as the <url-pattern/>. And then I have a Java Servlet class com.gaejexperiments.email.GAEJReceiveEmailServlet as the <servlet-class>. The security constraint has been added so that in case anyone invokes your url directly, then only Google Account authenticated users will be able to do that.

Now, all we have to do is write our Servlet. As mentioned, the incoming Email messages will be POSTed to our Servlet, so we need a simple doPost(…) implemented in our Servlet. The code is shown below:

package com.gaejexperiments.email;

import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;

import javax.mail.MessagingException;
import javax.mail.Multipart;
import javax.mail.Part;
import javax.mail.Session;
import javax.mail.internet.MimeMessage;
import javax.servlet.ServletException;
import javax.servlet.http.*;

@SuppressWarnings("serial")
public class GAEJReceiveEmailServlet extends HttpServlet {
 public static final Logger _log = Logger.getLogger(GAEJReceiveEmailServlet.class.getName());

 @Override
 public void doPost(HttpServletRequest req, HttpServletResponse resp)
 throws ServletException, IOException {

 try {

 Properties props = new Properties();
 Session session = Session.getDefaultInstance(props, null);
 MimeMessage message = new MimeMessage(session, req.getInputStream());

 //Extract out the important fields from the Mime Message
 String subject = message.getSubject();

 _log.info("Got an email. Subject = " + subject);

 String contentType = message.getContentType();
 _log.info("Email Content Type : " + contentType);

 printParts(message);
 //Parse out the Multiparts
 //Perform business logic based on the email
 }
 catch (Exception ex) {
 _log.log(Level.WARNING, "Failure in receiving email : " + ex.getMessage());
 }
 }

 private static void printParts(Part p) throws IOException, MessagingException {
 Object o = p.getContent();

 if (o instanceof String) {
 System.out.println("This is a String");
 System.out.println((String)o);
 }
 else if (o instanceof Multipart) {
 System.out.println("This is a Multipart");
 Multipart mp = (Multipart)o;

 int count = mp.getCount();
 for (int i = 0; i < count; i++) {
 printParts(mp.getBodyPart(i));
 }
 }
 else if (o instanceof InputStream) {
 System.out.println("This is just an input stream");
 InputStream is = (InputStream)o;
 int c;
 while ((c = is.read()) != -1)
 System.out.write(c);
 }
 }

}

Let us discuss the main parts of the code:

1. We have a doPost() method that gets invoked by the Google App Engine when an email is received.

2. In the doPost() method, we build out the email message (a instance of class MimeMessage) using the javax.mail.* classes as shown below:

Properties props = new Properties();
 Session session = Session.getDefaultInstance(props, null);
 MimeMessage message = new MimeMessage(session, req.getInputStream());

3. We extract out key attributes from the Message like subject, content type, etc.


//Extract out the important fields from the Mime Message
 String subject = message.getSubject();
 _log.info("Got an email. Subject = " + subject);

 String contentType = message.getContentType();
 _log.info("Email Content Type : " + contentType);

4. We have a utility method printParts(), that helps simply print out the contents of the message. But you could explore the Java Mail API to parse out the multiparts as required and then incorporate your business logic.
5. To help debug our Servlet, we have put in some log statements along with System Out statements, which we shall look for to verify that the Application did receive email.

Finally, we have used the INFO level to log if the message was sent out successfully or not, so we will have the change the logging level by modified the logging.properties file present in the war\WEB-INF folder. The necessary line after modification is shown below:

# Set the default logging level for all loggers to INFO
.level = INFO

Deploying our application

To deploy the application, you will need to first create your Application ID. The Application Identifier can be created by logging in at http://appengine.google.com with your Google Account. You will see a list of application identifiers already registered under your account (or none if you are just getting started). To create a new Application, click on the Create Application button and provide the Application Identifier as requested. Please note this name down since you will be using it for deployment.

For e.g. I have registered an application identifier named gaejexperiments.

To deploy the application, follow these steps (they should be familiar to you now):

  1. Click on the Deploy Icon in the Toolbar.
  2. In the Deploy dialog, provide your Email and Password. Do not click on Deploy button yet.
  3. Click on the App Engine Project settings link. This will lead you to a dialog, where you need to enter your Application ID [For e.g. my Application Identifier gaejexperiments]
  4. Click on OK. You will be lead back to the previous screen, where you can click on the Deploy button. This will start deploying your application to the GAEJ cloud. You should see several messages in the Console window as the application is being deployed.
  5. Finally, you should see the message “Deployment completed successfully”.

Testing our Application

To test our application, send an email from any mail client to an address within your application. As explained below, the email address to be used is shown below:

SomeID@YourApplicationId.appspotmail.com

For e.g. my Application Id is gaejexperiments, so I can send email to any of the following:

  • test@gaejexperiments.appspotmail.com
  • user1@gaejexperiments.appspotmail.com
  • and so on…

Once the email has been sent successfully from a Google Mail Account or Yahoo Mail or Outlook/Thunderbird, etc – you can use the App Engine console to verify if your application received the email or not. To do that, perform the following steps:

  1. Go to http://appengine.google.com and log in with your account.
  2. You will see a list of applications registered. Click on the application that you just deployed. In my case, it is gaejexperiments.
  3. When you click on a particular application, you will be taken to the Dashboard for that application, which contains a wealth of information around the requests, quotas, logs, versions, etc. This will be a subject of a future episode but for now, it is sufficient to say that you can come over here to monitor the health of your application and also to analyze what is going on.
  4. Click on the Logs link as shown in the screenshot below:console1
  5. This will display the application log. And all your application log statements that you code using the Logger class can be visible here.
  6. By default, the severity level is set at ERROR and we can change that to DEBUG, the lowest level and you should be able your log statements that had the log level of INFO. This was the log level at which we had logged statements like Received an Email, etc in our Java Servlet, so that is what we need to check for.
  7. If you see the statements, it means that the message has been received. Shown below is a screen shot of the log for a message that I sent.console2

Once this mechanism is in place, it means that your application has been correctly setup and deployed for receiving email messages. You can now build in business logic on what you need to do when an email is received.

Hope you had a good time reading this episode. The next episode I plan to extend an existing episode in which we built an XMPP Bot and deployed it on the Google App Engine. I plan to extend the bot to support the collaboration tool Google Wave, so that your bot can participate in a Wave conversation too. Till then, happy GAEJ’ing!

Advertisements
Categories: Uncategorized Tags:

Episode 3: Using the GAEJ Email Service

October 9, 2009 14 comments

Hope you have got going with the Google App Engine for Java (GAEJ) through the first two episodes where we got started with it in the first episode and then wrote our own XMPP bot in the second episode. In case you still need to get an introduction to the mechanics, please do not hesitate from reading the first episode.

In this episode, we are going to look at the Email Service in GAEJ. You can read the official documentation provided at the Google site over here. We will be keeping this episode simple but the key to take away from this episode is the ease with which you can integrate email sending into your GAEJ hosted applications. Please note that the service currently provides the ability to only send emails/attachments, etc. There is no provision at this point of time for your GAEJ hosted application to receive emails.

Our approach to this episode will be consistent with the earlier ones where we will follow these steps:

  • See the application work first
  • Create a new project and write our code that utilizes the Email Service
  • Deploy and see it work

Email Service in Action

My approach in this and forthcoming episodes will be to focus on the Server side architecture and not much on the User Interface. What it means is that I could have demonstrated the Email Service by coding up a simple HTML form that accepts the To , Subject and Body of the Email to be sent and then it will hit a Server side script hosted in your GAEJ application and send off the email. But instead, I will assume that you are comfortable with whatever client (HTML, Flex, Java, etc) programming language and simply need a way to figure out what to do on the Server side. Hence we will focus on the Server side implementation.

To see the Email Service in Action, I have already the hosted the application that I wrote. The application id is gaejexperiments and hence available at http://gaejexperiments.appspot.com. In case you are still not clear on the Application Id, I suggest you read the first episode here.

To send an email, we will use a REST like style to invoke the Email Service. This is nothing much a servlet that has been hosted in my gaejexperiments application. To invoke the Email Service, all you need to do is punch in the following line in your favourite browser (IE, Firefox, Chrome, Safari,etc) and press enter:

http://gaejexperiments.appspot.com/gaejemail?email_to=%5BYourEmailId%5D&email_subject=%5BEmailSubject%5D&email_body=%5BEmailBody%5D

You need to substitute the values [YourEmailId],[EmailSubject] and [EmailBody] with your email id, a sample subject and a sample body (message text). Please make sure to encode your text as appropriate in case you have special characters, whitespaces, etc.

An example is shown here:

http://gaejexperiments.appspot.com/gaejemail?email_to=romin.k.irani@gmail.com&email_subject=Hi&email_body=Test

This should send out an email (on my behalf) and you will see a plain text message returned by our Service, which will indicate if it succeeded or failed in sending out the email. The sender of the message is coded in the service itself and we are not giving a provision here to specify that. It is just the way that I have designed the service and you can do it your way.

Remember that our application delivers the message onto the Google Infrastructure to deliver the message and then it will take over and try to deliver the message. That is pretty much standard stuff.

Develop our Project and utilize the Email Service

The first thing to do is to create a New Google Web Application Project. Follow these steps:

1. Either click on File –> New –> Other or press Ctrl-N to create a new project. Select Google and then Web Application project. Alternately you could also click on the New Web Application Project Toolbar icon as part of the Google Eclipse plugin.
2. In the New Web Application Project dialog, deselect the Use Google Web Toolkit and give a name to your project. I have named mine GAEJExperiments. I suggest you go with the same name so that things are consistent with the rest of the article, but I leave that to you.
3. Click on Finish

This will generate the project and also create a sample Hello World Servlet for you. But we will be writing our own Servlet.

Coding the GAEJEmailServlet.java

Create a new Servlet in your Project as shown below. I have created the GAEJEmailServlet.java in the package com.gaejexperiments.email. You can choose a package of your choice.  The code is straightforward and is listed below:


package com.gaejexperiments.email;

import java.io.IOException;
import java.util.Properties;

import javax.mail.Message;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import javax.servlet.ServletException;
import javax.servlet.http.*;

@SuppressWarnings("serial")
public class GAEJEmailServlet extends HttpServlet {
 public void doGet(HttpServletRequest req, HttpServletResponse resp)
 throws IOException {

 String strCallResult = "";
 resp.setContentType("text/plain");
 try {
 //Extract out the To, Subject and Body of the Email to be sent
 String strTo = req.getParameter("email_to");
 String strSubject = req.getParameter("email_subject");
 String strBody = req.getParameter("email_body");

 //Do validations here. Only basic ones i.e. cannot be null/empty
 //Currently only checking the To Email field
 if (strTo == null) throw new Exception("To field cannot be empty.");

 //Trim the stuff
 strTo = strTo.trim();
 if (strTo.length() == 0) throw new Exception("To field cannot be empty.");

 //Call the GAEJ Email Service
 Properties props = new Properties();
 Session session = Session.getDefaultInstance(props, null);
 Message msg = new MimeMessage(session);
 msg.setFrom(new InternetAddress("#YOUR EMAIL ADDRESS HERE#"));
 msg.addRecipient(Message.RecipientType.TO,
 new InternetAddress(strTo));
 msg.setSubject(strSubject);
 msg.setText(strBody);
 Transport.send(msg);
 strCallResult = "Success: " + "Email has been delivered.";
 resp.getWriter().println(strCallResult);
 }
 catch (Exception ex) {
 strCallResult = "Fail: " + ex.getMessage();
 resp.getWriter().println(strCallResult);
 }
 }

 @Override
 public void doPost(HttpServletRequest req, HttpServletResponse resp)
 throws ServletException, IOException {
 doGet(req, resp);
 }

}

Let us go through the main pieces of the code:

  1. I have provided both GET and POST handlers in the servlet and the POST handler simply invokes the GET handler here.
  2. Then we parse the request parameters for the recipient email address (email_to) and the subject and body of the email i.e. email_subject and email_body respectively.
  3. We do some basic validation on the email_to field to make sure that it is not empty. I am currently not doing at validation on the subject and body fields. Additionally, you can easily modify this to receive more than one email address and loop through them yourself.
  4. Finally we have the block of code that sends email. This is pretty much standard JavaMail stuff that is used over here.  We get a mail session object. Then we create the MimeMessage in which we can populate the sender (from Email id), the recipient and finally the subject and body. We use the Transport object to send the email out and return back a plain text string indicating success or failure in delivering the email to the transport.
  5. Please note that the sender email id needs to be populated by you. I have intentionally left it as #YOUR EMAIL ADDRESS HERE# in the code above.

Configuring the Servlet in web.xml file

To complete our Servlet development, we will also need to add the <servlet/> and <servlet-mapping/> entry to the web.xml file. This file is present in the WEB-INF folder of the project. The necessary fragment to be added to your web.xml file are shown below. Please note that you can use your own namespace and servlet class. Just modify it accordingly if you do so.


<servlet>
 <servlet-name>GAEJEmailServlet</servlet-name>
 <servlet-class>com.gaejexperiments.email.GAEJEmailServlet</servlet-class>
 </servlet>
 <servlet-mapping>
 <servlet-name>GAEJEmailServlet</servlet-name>
 <url-pattern>/gaejemail</url-pattern>
 </servlet-mapping>

Deploy the Application and utilize it

To deploy the application, you will need to first create your Application ID. The Application Identifier can be created by logging in at http://appengine.google.com with your Google Account. You will see a list of application identifiers already registered under your account (or none if you are just getting started). To create a new Application, click on the Create Application button and provide the Application Identifier as requested. Please note this name down since you will be using it for deployment.

For e.g. I have registered an application identifier named gaejexperiments.

To deploy the application, follow these steps (they should be familiar to you now):

  1. Click on the Deploy Icon in the Toolbar.
  2. In the Deploy dialog, provide your Email and Password. Do not click on Deploy button yet.
  3. Click on the App Engine Project settings link. This will lead you to a dialog, where you need to enter your Application ID [For e.g. my Application Identifier gaejexperiments]
  4. Click on OK. You will be lead back to the previous screen, where you can click on the Deploy button. This will start deploying your application to the GAEJ cloud. You should see several messages in the Console window as the application is being deployed.
  5. Finally, you should see the message “Deployment completed successfully”.

This means that you application is ready to serve and you can verify the Email service by invoking the appropriate url which you can punch into your browser. The format will be the following:

http://[YourApplicationId].appspot.com/gaejemail?email_to=%5BYourEmailId%5D&email_subject=%5BEmailSubject%5D&email_body=%5BEmailBody%5D

Moving forward

In this episode, we have seen how to utilize the Email Service provided by the Google App Engine. The code that we covered is straightforward to implement in your application. Do note once again that the Email Service currently does *not allow* for functionality that can receive emails. You can study the JavaMail API in more detail to understand how to do things like sending attachments in your email too. You can do that using the Google App Engine Email Service.

The other key thing to take away from this episode is also a focus on the server side api rather than building HTML front-ends, etc. I leave that to the user to design and implement since invoking a standards based HTTP GET or POST is now possible in almost any client programming platform.

In the next episode, we shall look at the Networking API available under GAEJ. The focus will be on invoking extrernal URLs and consuming external Web Services or even RSS feeds.

Till the next episode, good bye and happy coding!

Categories: Uncategorized Tags: