Posts

Image
String Encoding for Talend/JaspersoftETL for Basic Authentication/md5/sha-256 What is a convenient way to encode usernames/passwords for use in my ETL jobs? First copy the java code on this Github Gist into your clipboard: Second  Create a routine called hashingUtils (name is important!). Paste the Java code from above and save the routine Third you can now use the following methods in any component: hashingUtils.basicAuth("username","password") hashingUtils.md5encode("string") hashingUtils.sha256encode("string") For example, in a tHttpRequest Component:

Dealing with Nested Documents in MongoDB and Talend (aka Baking a Cake)

Image
Abstract: MongoDB is a very popular document database that gives you a huge amount of flexibility when it comes to storing data. On the other hand, the traditional relational model is far less flexible in how data is stored - you're limited to columns and rows. Sometimes you want to go from a flexible model like MongoDB to a relational one, that's what this post attempts to explain using  Talend / JaspersoftETL  (ETL tools). I do not want to get into the relational vs non-relational model argument in this post - it's only an example if you need to do this for some reason...I'm pro choice :) Scenario: We have a JSON document in MongoDB that looks like this: { "id" : "0001", "type" : "donut", "name" : "Cake", "ppu" : 0.55, "batters" : { "batter" : [ ...

Deleting Old Scheduled Report Ouptuts Using the Jaspersoft REST API

Image
I get this request quite a bit - how do I clean up old artifacts from the Jaspersoft repository? Old artifacts are typically reports that are scheduled and then saved into the repository. I explain this in a short video: Here are some specifics about the Talend/Jaspersoft ETL job used: tHttpRequest - Does a GET to the URI "http://localhost:8080/jasperserver-pro/rest_v2/resources?recursive=true&type=file&folderUri=/public/ScheduledReports" and writes it to an XML file. API reference- see "Searching the Repository" tFileInputXML - Reads the XML response from file (written by tHttpRequest) tFilterRow - Filters the XML file on the field "creationDate" with the value: TalendDate.parseDate("yyyy-MM-dd", "2014-03-04") tFlowToIterate - takes each request and iterates through them tREST - For each file (URI) we run the DELETE command: "http://localhost:8080/jasperserver-pro/rest_v2/resources" + ((String)globalMap....

Geospatial Features in MongoDB with Jaspersoft

Image
Lately I have been posting technical examples to the Jaspersoft community wiki instead of this blog. I'll cross-post this one because there are not many examples of doing this. When working with Geospatial data, I found the $near operator to be pretty sweet. Let me show you by example. In the screenshots below you'll see that I constructed a report that shows "cases" on a map. When you click on a specific case you will be taken to a detail level report. The detail level report will show you cases nearby for that service. So for example, if someone reports a broken street light on Market Street, this might help you locate similar cases. So how is it done?  Quite simple really, take a look at this mongodb  query : Line 8: The $P{location} parameter is a java.util.List that contains longitude and latitude coordinates, something like [-122.4194200,37.7749300] Line 11: How many meters from the center (specified in line 8) should it search Line 13: T...

New MongoDB Components in Talend/JaspersoftETL

Image
I uploaded two videos with examples of using the tMongoDBInput and tMongoDBRow components that are shipping with JaspersoftETL 5.2.2 (Plus edition) and Talend 5.2.2+ I recommend you change the quality to 720p to watch the videos in better quality. I hope you find them useful! Part 1: Using tMongoDBInput (and using Talend's built in aggregation engine): Part 2: Using tMongoDBRow (and the aggregation framework):
Emailing me files is no longer OK. It's 2013 - wiki, evernote, soundcloud, google docs, etc. Files in 2013 are like paper in 1999 — ernestbro (@not_a_poet) February 28, 2013

Report Bursting in JasperReports Server

Image
What is report bursting? Take this example. You have a report that shows all the accounts that a particular employee owns in your company. The user can log into your reporting system and ask for the report, he'll be asked to enter his employee ID or maybe it will be filled in automatically based on who he/she is. Now your boss asks you (a BI professional) to send each employee their list of accounts on a weekly basis...yikes! You need to automate this. What do you do? As of Jaspersoft version 5.0 there's no UI built into the web-application to handle the complex task of report bursting. Designing an UI that let's you iterate through any possibility of inputs, read any possibility of values and output to any possible medium is tricky, for now you have to build your own utilities using the rich SOAP or REST APIs that are available in JasperReports Server. The easiest way I found to do this using is using JaspersoftETL (Talend) and a component called tJasperServerExec ...