tag:blogger.com,1999:blog-60212561695058524272024-03-21T15:49:09.600-07:00tech poetA personal blog about technologyErnesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.comBlogger51125tag:blogger.com,1999:blog-6021256169505852427.post-81986774702243874072015-05-08T10:06:00.000-07:002015-05-08T10:35:28.380-07:00<div dir="ltr" style="text-align: left;" trbidi="on">
<h2>
<span lang="EN-GB">String Encoding for Talend/JaspersoftETL for Basic Authentication/md5/sha-256</span></h2>
<h3 style="text-align: left;">
<span lang="EN-GB">What is a convenient way to encode usernames/passwords for use in my ETL jobs?</span></h3>
<div class="MsoNormal">
<span lang="EN-GB"><b>First</b> copy the java code on this Github Gist into your
clipboard:<o:p></o:p></span></div>
<script src="https://gist.github.com/ernestoongaro/1a36b95a663b6b0cc54a.js"></script>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><b><br /></b></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><b>Second</b> Create a routine called
hashingUtils (name is important!). Paste the Java code from above and save the
routine<o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpeHZhQ577ixvWVLsMHQ8v7Oto_Vhj64g6GNBQscku0gBGl1n_9qGzvVtA7S0ACUWyk1RNlONCHV1eFTqCXANDpOQtGkplnAN-fUNnFQNjGKd37pX-hR9yH1GVV0TyFbqbr5fHZQYxmVc/s1600/routine.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="136" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpeHZhQ577ixvWVLsMHQ8v7Oto_Vhj64g6GNBQscku0gBGl1n_9qGzvVtA7S0ACUWyk1RNlONCHV1eFTqCXANDpOQtGkplnAN-fUNnFQNjGKd37pX-hR9yH1GVV0TyFbqbr5fHZQYxmVc/s200/routine.png" width="200" /></a></div>
<div class="MsoNormal">
<span style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><!--[if gte vml 1]><v:shapetype
id="_x0000_t75" coordsize="21600,21600" o:spt="75" o:preferrelative="t"
path="m@4@5l@4@11@9@11@9@5xe" filled="f" stroked="f">
<v:stroke joinstyle="miter"/>
<v:formulas>
<v:f eqn="if lineDrawn pixelLineWidth 0"/>
<v:f eqn="sum @0 1 0"/>
<v:f eqn="sum 0 0 @1"/>
<v:f eqn="prod @2 1 2"/>
<v:f eqn="prod @3 21600 pixelWidth"/>
<v:f eqn="prod @3 21600 pixelHeight"/>
<v:f eqn="sum @0 0 1"/>
<v:f eqn="prod @6 1 2"/>
<v:f eqn="prod @7 21600 pixelWidth"/>
<v:f eqn="sum @8 21600 0"/>
<v:f eqn="prod @7 21600 pixelHeight"/>
<v:f eqn="sum @10 21600 0"/>
</v:formulas>
<v:path o:extrusionok="f" gradientshapeok="t" o:connecttype="rect"/>
<o:lock v:ext="edit" aspectratio="t"/>
</v:shapetype><v:shape id="Picture_x0020_3" o:spid="_x0000_i1026" type="#_x0000_t75"
style='width:171pt;height:116pt;visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file://localhost/Users/eongaro/Library/Caches/TemporaryItems/msoclip/0/clip_image001.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><span lang="EN-GB"><o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b>Third</b> you can now use the following methods
in any component:</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-family: "Andale Mono";">hashingUtils.basicAuth("username","password")<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-family: "Andale Mono";">hashingUtils.md5encode("string")<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="font-family: "Andale Mono";">hashingUtils.sha256encode("string")<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-GB">For example, in a tHttpRequest Component:<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="mso-ansi-language: EN-US; mso-no-proof: yes;"><!--[if gte vml 1]><v:shape
id="Picture_x0020_4" o:spid="_x0000_i1025" type="#_x0000_t75" style='width:6in;
height:259pt;visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file://localhost/Users/eongaro/Library/Caches/TemporaryItems/msoclip/0/clip_image003.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><span lang="EN-GB"><o:p></o:p></span></div>
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]--><!--[if gte mso 9]><xml>
<o:DocumentProperties>
<o:Revision>0</o:Revision>
<o:TotalTime>0</o:TotalTime>
<o:Pages>1</o:Pages>
<o:Words>96</o:Words>
<o:Characters>552</o:Characters>
<o:Company>Jaspersoft</o:Company>
<o:Lines>4</o:Lines>
<o:Paragraphs>1</o:Paragraphs>
<o:CharactersWithSpaces>647</o:CharactersWithSpaces>
<o:Version>14.0</o:Version>
</o:DocumentProperties>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]-->
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves>false</w:TrackMoves>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-US</w:LidThemeOther>
<w:LidThemeAsian>JA</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
<w:UseFELayout/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="276">
<w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" Priority="39" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" Name="toc 9"/>
<w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
</w:LatentStyles>
</xml><![endif]-->
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:Cambria;
mso-ascii-font-family:Cambria;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Cambria;
mso-hansi-theme-font:minor-latin;}
</style>
<![endif]-->
<!--StartFragment-->
<!--EndFragment--><br />
<div class="MsoNormal">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj60gp1vuZGXiM6BS_phDd6cM5WVvJvJ5WNGlqzIZw_nfqRB-XgaWlo7c5T8iu2meLrCEl94foYrZkXZB8ErSvCiw8a9-GzCx32mKFteTy0rDRip2CCd-XTts19T7_-M-MkGlFyM8GG6o/s1600/example.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj60gp1vuZGXiM6BS_phDd6cM5WVvJvJ5WNGlqzIZw_nfqRB-XgaWlo7c5T8iu2meLrCEl94foYrZkXZB8ErSvCiw8a9-GzCx32mKFteTy0rDRip2CCd-XTts19T7_-M-MkGlFyM8GG6o/s400/example.png" width="400" /></a></div>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com2tag:blogger.com,1999:blog-6021256169505852427.post-92214989464252933402014-06-06T04:46:00.000-07:002017-04-07T05:23:21.229-07:00Dealing with Nested Documents in MongoDB and Talend (aka Baking a Cake)<div dir="ltr" style="text-align: left;" trbidi="on">
<h4 style="text-align: left;">
Abstract:</h4>
<div>
<a href="http://mongodb.com/">MongoDB</a> is a very popular document database that gives you a huge amount of flexibility when it comes to storing data. On the other hand, the traditional relational model is far less flexible in how data is stored - you're limited to columns and rows. Sometimes you want to go from a flexible model like MongoDB to a relational one, that's what this post attempts to explain using <a href="http://talend.com/">Talend</a>/<a href="http://jaspersoft.com/">JaspersoftETL</a> (ETL tools).<br />
<br />
I do not want to get into the relational vs non-relational model argument in this post - it's only an example if you need to do this for some reason...I'm pro choice :)</div>
<div>
<br /></div>
<h4 style="text-align: left;">
Scenario:</h4>
<div>
We have a JSON document in MongoDB that looks like this:</div>
<pre style="background-color: #f0f0f0; background-position: initial initial; background-repeat: initial initial; border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> {
</code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "0001",
</code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "donut",
</code><code style="word-wrap: normal;"><span style="color: blue;">"name"</span></code><code style="color: black; word-wrap: normal;">: "Cake",
</code><code style="word-wrap: normal;"><span style="color: blue;">"ppu"</span></code><code style="color: black; word-wrap: normal;">: 0.55,
</code><code style="word-wrap: normal;"><span style="color: blue;">"batters"</span></code><code style="color: black; word-wrap: normal;">:
{
</code><code style="word-wrap: normal;"><span style="color: blue;"> "batter"</span></code><code style="color: black; word-wrap: normal;">:
[
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "1001", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Regular" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "1002", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Chocolate" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "1003", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Blueberry" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "1004", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Devil's Food" }
]
},
</code><code style="word-wrap: normal;"><span style="color: blue;">"topping"</span></code><code style="color: black; word-wrap: normal;">:
[
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "5001", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "None" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "5002", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Glazed" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "5005", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Sugar" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "5007", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Powdered Sugar" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "5006", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Chocolate with Sprinkles" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "5003", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Chocolate" },
{ </code><code style="word-wrap: normal;"><span style="color: blue;">"id"</span></code><code style="color: black; word-wrap: normal;">: "5004", </code><code style="word-wrap: normal;"><span style="color: blue;">"type"</span></code><code style="color: black; word-wrap: normal;">: "Maple" }
]
}
</code></pre>
<div>
We want to put the document into three different tables, for use in a relational database:<br />
<h4 style="text-align: left;">
Main Table:</h4>
<table>
<tbody>
<tr>
<th>ID</th>
<th>TYPE</th>
<th>NAME</th>
<th>PPU</th>
</tr>
<tr>
<td>0001</td>
<td>donut</td>
<td>cake</td>
<td>0.55</td>
</tr>
</tbody></table>
<h4 style="text-align: left;">
Batter Table:</h4>
<table>
<tbody>
<tr>
<th>BATTERID</th>
<th>BATTERTYPE</th>
<th>CAKEID</th>
</tr>
<tr>
<td>1001</td>
<td>regular</td>
<td>0001</td>
</tr>
<tr>
<td>1002</td>
<td>chocolate</td>
<td>0001</td>
</tr>
<tr>
<td>1003</td>
<td>blueberry</td>
<td>0001</td>
</tr>
<tr>
<td>1004</td>
<td>devil's food</td>
<td>0001</td>
</tr>
</tbody></table>
<h4 style="text-align: left;">
Topping Table:</h4>
<table>
<tbody>
<tr>
<th>TOPPINGID</th>
<th>TOPPINGTYPE</th>
<th>CAKEID</th>
</tr>
<tr>
<td>5001</td>
<td>None</td>
<td>0001</td>
</tr>
<tr>
<td>5002</td>
<td>Glazed</td>
<td>0001</td>
</tr>
<tr>
<td>5005</td>
<td>Sugar</td>
<td>0001</td>
</tr>
<tr>
<td>5007</td>
<td>Powdered Sugar</td>
<td>0001</td>
</tr>
<tr>
<td>5006</td>
<td>Chocolate with Sprinkles</td>
<td>0001</td>
</tr>
<tr>
<td>5003</td>
<td>Chocolate</td>
<td>0001</td>
</tr>
<tr>
<td>5004</td>
<td>Maple</td>
<td>0001</td>
</tr>
</tbody></table>
</div>
<h4 style="text-align: left;">
Talend tMongoDBInput Component</h4>
<div>
The tMongoDBInput component is what we'll use to extract the data from MongoDB and input into Talend. This is fairly straightforward, you can configure the connection and then you will have to map some fields. We map id, type, name, ppu as regular fields. The batter and topping fields are somewhat special. </div>
<div>
<br /></div>
<div>
<b>Toppings:</b></div>
<div>
The toppings field will be mapped a string and will contain an object in each row that we'll need to deal with later</div>
<div>
<br /></div>
<div>
<b>Batters:</b></div>
<div>
Batters is a nested document within another nested document and we'll need to define a parent node path. tMongoDBInput does not allow you to use node notation to select fields (like batters.batter) so you'll define a parent node like this:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieLV2vRdCDJ8LHLDahzVMk0POgLsvvOR-_v7R8QYCnbv4IteI4TIvCdHAg_3hwUOdhvWkg32a1R9tH17XJd69qBwsSta1qEk07ae4viKMCaCUOUbcFDi7FCWAdiKyVpFWyNq2h3IX__OQ/s1600/Untitled.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="66" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieLV2vRdCDJ8LHLDahzVMk0POgLsvvOR-_v7R8QYCnbv4IteI4TIvCdHAg_3hwUOdhvWkg32a1R9tH17XJd69qBwsSta1qEk07ae4viKMCaCUOUbcFDi7FCWAdiKyVpFWyNq2h3IX__OQ/s1600/Untitled.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
In the end the schema for tMongoDBInput will look like this:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLOje-OSfFoptvPcIBIRDXu3GK7lm_NEqglZ-Xq0s8Zfon2ggeVYQKWew5rfXTOd2EnsFBX6Ydf3ZmhhANI2z0xbw-UEQhSPHisSaCxY4FTY4Ul2DFbMODYMPR7EI0OvfOE-FL4oVB9eg/s1600/Screen+Shot+2014-06-06+at+12.28.40+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="234" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLOje-OSfFoptvPcIBIRDXu3GK7lm_NEqglZ-Xq0s8Zfon2ggeVYQKWew5rfXTOd2EnsFBX6Ydf3ZmhhANI2z0xbw-UEQhSPHisSaCxY4FTY4Ul2DFbMODYMPR7EI0OvfOE-FL4oVB9eg/s1600/Screen+Shot+2014-06-06+at+12.28.40+PM.png" width="320" /></a></div>
<h4 style="text-align: left;">
tMap Component - Splitting</h4>
<div class="separator" style="clear: both; text-align: left;">
Now we'll use a tMap component to split the outputs into three (main, topping, batter)</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfahs51R9HX_O2MocOOuMI_ZRXo2rheFv_eCzvTCTwS_BeTIWv1oG_ahPk31ajpM8jDUY7qdob_VSMONuV6yjmlwwQMMJfGqaRP5Co0iZqCOKoSJccNOorIjfOXY0-VC_8ZM5EuX0hgxc/s1600/Screen+Shot+2014-06-06+at+12.32.50+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfahs51R9HX_O2MocOOuMI_ZRXo2rheFv_eCzvTCTwS_BeTIWv1oG_ahPk31ajpM8jDUY7qdob_VSMONuV6yjmlwwQMMJfGqaRP5Co0iZqCOKoSJccNOorIjfOXY0-VC_8ZM5EuX0hgxc/s1600/Screen+Shot+2014-06-06+at+12.32.50+PM.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div class="p1">
<b>tExtractJSONFields - Expanding Documents</b><br />
So now we have to deal with the special fields, batter and topping - each one of them contains a list that we want to denormalize into it's own rows. We can use the tExtractJSONFields component for this. The component has a couple of key things: schema, Loop XPath Query and XPath query. This determines what fields to map.<br />
<br />
<b>Schema:</b><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP_f9MwY3lbgUH64V2K1LQBRCZOtrXjOHQQjVgn_JfvDf2HLrSq9DAtXHIzvQ0b-isUBRLD-m6nwKlItkYwa4tCTMuAYR7O4QTfmJpR6OAv-BRUfrhkSw6OXB08qUrMBiRT3P1gxtCpK4/s1600/Screen+Shot+2014-06-06+at+12.36.05+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP_f9MwY3lbgUH64V2K1LQBRCZOtrXjOHQQjVgn_JfvDf2HLrSq9DAtXHIzvQ0b-isUBRLD-m6nwKlItkYwa4tCTMuAYR7O4QTfmJpR6OAv-BRUfrhkSw6OXB08qUrMBiRT3P1gxtCpK4/s1600/Screen+Shot+2014-06-06+at+12.36.05+PM.png" width="640" /></a></div>
<br />
<b>Then set the Xpath queries:</b><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv9fbwt9ozQ-s2bixgMweh5x3Z_i5hJgZnZDM5h75aS5hTQkq6h6c5GmsaSJ34aayPoNSh_vHd77kPl6efhcX8fe_kExCJALvQWakea_9U761v5B9a7XpYqGmIhkjwvg1vE5vlD8jzoFw/s1600/Screen+Shot+2014-06-06+at+12.37.57+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv9fbwt9ozQ-s2bixgMweh5x3Z_i5hJgZnZDM5h75aS5hTQkq6h6c5GmsaSJ34aayPoNSh_vHd77kPl6efhcX8fe_kExCJALvQWakea_9U761v5B9a7XpYqGmIhkjwvg1vE5vlD8jzoFw/s1600/Screen+Shot+2014-06-06+at+12.37.57+PM.png" width="640" /></a></div>
<br />
And finally glue it all together:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhk0NGvEv0oJTcoEKkRHRzYxvQn5taeZMaaqfDZp8FT9uUiCe7B_roE0mZv-_5scoR4EIZJip8dZF-k1S9L7eL8w-OTwa7C6VaHJEyHntLsV4Da1SCk3DlUdPRKe7ujqGJQxefrLmbzNmU/s1600/Screen+Shot+2014-06-06+at+12.38.58+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="418" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhk0NGvEv0oJTcoEKkRHRzYxvQn5taeZMaaqfDZp8FT9uUiCe7B_roE0mZv-_5scoR4EIZJip8dZF-k1S9L7eL8w-OTwa7C6VaHJEyHntLsV4Da1SCk3DlUdPRKe7ujqGJQxefrLmbzNmU/s1600/Screen+Shot+2014-06-06+at+12.38.58+PM.png" width="640" /></a></div>
Note that we have 3 input rows, then for each batter and topping we are extracting.<br />
<br />
<h4 style="text-align: left;">
If you are interested in this example, you can download it here:</h4>
<div>
<ul style="text-align: left;">
<li><a href="https://www.dropbox.com/s/7vvtl328xw7tew9/doc.json">Data</a>: This is a JSON object you can import with mongoimport (like <i>mongoimport --jsonArray --collection nestedSample < doc.json</i>)</li>
<li><a href="https://www.dropbox.com/s/y3jtfcrp0vwcywp/TalendJob.zip">JaspersoftETL Job</a>: This job works with JaspersoftETL 5.4.1 Big Data Edition (or presumably the Open Source Big Data edition of Talend)</li>
</ul>
</div>
</div>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com12tag:blogger.com,1999:blog-6021256169505852427.post-21409339289287319692014-04-04T08:53:00.001-07:002014-04-04T08:53:02.893-07:00Deleting Old Scheduled Report Ouptuts Using the Jaspersoft REST API<div dir="ltr" style="text-align: left;" trbidi="on">
I get this request quite a bit - how do I clean up old artifacts from the Jaspersoft repository? Old artifacts are typically reports that are scheduled and then saved into the repository. I explain this in a short video:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='420' height='366' src='https://www.youtube.com/embed/NhsZecdsLYs?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
Here are some specifics about the Talend/Jaspersoft ETL job used:<br />
<ul style="text-align: left;">
<li>tHttpRequest - Does a GET to the URI "http://localhost:8080/jasperserver-pro/rest_v2/resources?recursive=true&type=file&folderUri=/public/ScheduledReports" and writes it to an XML file. <a href="http://community.jaspersoft.com/documentation/jasperreports-server-web-services-guide/v550/v2resources-service">API reference- see "Searching the Repository"</a></li>
<li>tFileInputXML - Reads the XML response from file (written by tHttpRequest)</li>
<li>tFilterRow - Filters the XML file on the field "creationDate" with the value: TalendDate.parseDate("yyyy-MM-dd", "2014-03-04")</li>
<li>tFlowToIterate - takes each request and iterates through them</li>
<li>tREST - For each file (URI) we run the DELETE command: "http://localhost:8080/jasperserver-pro/rest_v2/resources" + ((String)globalMap.get("row2.uri")) <a href="http://community.jaspersoft.com/documentation/jasperreports-server-web-services-guide/v550/v2resources-service">API reference- see "Deleting Resources"</a></li>
</ul>
<div>
You can find my example job <a href="https://drive.google.com/file/d/0BxbdWGHYkr_hZEJ6UUNJb29hMG8/edit?usp=sharing">here</a> - it is for Talend 5.4.1 </div>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com1tag:blogger.com,1999:blog-6021256169505852427.post-51677821122033423532013-12-01T09:09:00.001-08:002013-12-06T06:54:58.157-08:00Geospatial Features in MongoDB with Jaspersoft<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
Lately I have been posting technical examples to the Jaspersoft community wiki instead of this blog. I'll cross-post this one because there are not many examples of doing this. When working with Geospatial data, I found the $near operator to be pretty sweet. Let me show you by example.<br />
<br />
In the screenshots below you'll see that I constructed a report that shows "cases" on a map. When you click on a specific case you will be taken to a detail level report. The detail level report will show you cases nearby for that service. So for example, if someone reports a broken street light on Market Street, this might help you locate similar cases.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhZAazRLIJMi9MFdafIVRX_Cbhf-YZcueeJR_1JWGtMM7K75pE3-2fo8MTMZlSEfpfCMp_K5JPlGlcfknLvuo3t_SlSPxo9JlsOr5UOxkq7LpDtZXry8YpsgDtd1f2QFbPn9YfGdHGHtA/s1600/Screen+Shot+2013-12-01+at+4.31.44+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="245" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhZAazRLIJMi9MFdafIVRX_Cbhf-YZcueeJR_1JWGtMM7K75pE3-2fo8MTMZlSEfpfCMp_K5JPlGlcfknLvuo3t_SlSPxo9JlsOr5UOxkq7LpDtZXry8YpsgDtd1f2QFbPn9YfGdHGHtA/s400/Screen+Shot+2013-12-01+at+4.31.44+PM.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcip6_F-guTqJam_cY2LGc-yE57opPOWYVtJ7khDvWbZfhQe5AgxCv3IaohW4L1KBbgnbiWzcDES1J0vDU393Is0I913tP5raYDgHAqDN2AG209fSMrY1yfW3vXyofi0jvsjFjdZ0lZYs/s1600/Screen+Shot+2013-12-01+at+4.28.52+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="245" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcip6_F-guTqJam_cY2LGc-yE57opPOWYVtJ7khDvWbZfhQe5AgxCv3IaohW4L1KBbgnbiWzcDES1J0vDU393Is0I913tP5raYDgHAqDN2AG209fSMrY1yfW3vXyofi0jvsjFjdZ0lZYs/s400/Screen+Shot+2013-12-01+at+4.28.52+PM.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<b>So how is it done?</b></div>
<div class="separator" style="clear: both; text-align: left;">
Quite simple really, take a look at this mongodb <a href="https://gist.github.com/anonymous/7736499">query</a>:
<script src="https://gist.github.com/anonymous/7736499.js"></script>
</div>
<br />
<br />
<br />
<br />
<li><b>Line 8:</b> The $P{location} parameter is a java.util.List that contains longitude and latitude coordinates, something like [-122.4194200,37.7749300]
</li>
<li><b>Line 11:</b> How many meters from the center (specified in line 8) should it search</li>
<li><b>Line 13:</b> The $P{service} parameter is a string containing the name of the Service, in the screenshot above it's Missing Sign</li>
<li>You will need to have an index on your location field, like so: <span style="font-family: Arial; font-size: 13px;">ensureIndex({"location" : "2dsphere" } )</span></li>
</div>
<div>
<br />
That's it, I've never really done anything with geospatial data and this seemed like it was a piece of cake.<br />
<h3 style="text-align: left;">
References:</h3>
</div>
<div>
<ul style="text-align: left;">
<li>Download the example data and reports from the <a href="http://community.jaspersoft.com/wiki/mongodb-city-cases-example">Jaspersoft Community Wiki</a></li>
<li>Read about the <a href="http://docs.mongodb.org/manual/reference/operator/query/near/">MongoDB $near operator</a></li>
<li>Watch me <a href="http://www.jaspersoft.com/watch/Reporting-Dashboards-Analytics-Big%20Data">present on this subject</a></li>
</ul>
</div>
<br /></div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-43850498213964940512013-06-27T04:25:00.002-07:002013-09-13T02:54:40.158-07:00New MongoDB Components in Talend/JaspersoftETL<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: left;">
I uploaded two videos with examples of using the tMongoDBInput and tMongoDBRow components that are shipping with JaspersoftETL 5.2.2 (Plus edition) and Talend 5.2.2+</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I recommend you change the quality to 720p to watch the videos in better quality. I hope you find them useful!</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Part 1: Using tMongoDBInput (and using Talend's built in aggregation engine):</div>
<iframe allowfullscreen="" frameborder="0" height="390" src="//www.youtube.com/embed/wtAv1WJYzxg" width="420"></iframe>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Part 2: Using tMongoDBRow (and the aggregation framework):</div>
<iframe allowfullscreen="" frameborder="0" height="390" src="//www.youtube.com/embed/x6C1SsVY_2I" width="420"></iframe>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com4Dublin, Ireland53.3498053 -6.260309699999993453.0463133 -6.9057566999999933 53.6532973 -5.6148626999999935tag:blogger.com,1999:blog-6021256169505852427.post-91680521415187210442013-02-28T02:57:00.002-08:002013-02-28T02:57:58.563-08:00<div dir="ltr" style="text-align: left;" trbidi="on">
<blockquote class="twitter-tweet">
Emailing me files is no longer OK. It's 2013 - wiki, evernote, soundcloud, google docs, etc. Files in 2013 are like paper in 1999<br />
— ernestbro (@not_a_poet) <a href="https://twitter.com/not_a_poet/status/307078991225958401">February 28, 2013</a></blockquote>
<script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script></div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-18537467848788358682013-01-02T09:27:00.002-08:002013-09-13T02:54:54.962-07:00Report Bursting in JasperReports Server<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="text-align: left;">
What is report bursting?</h2>
Take this example. You have a report that shows all the accounts that a particular employee owns in your company. The user can log into your reporting system and ask for the report, he'll be asked to enter his employee ID or maybe it will be filled in automatically based on who he/she is.<br />
<br />
Now your boss asks you (a BI professional) to send each employee their list of accounts on a weekly basis...yikes! You need to automate this. What do you do?<br />
<br />
As of Jaspersoft version 5.0 there's no UI built into the web-application to handle the complex task of report bursting. Designing an UI that let's you iterate through any possibility of inputs, read any possibility of values and output to any possible medium is tricky, for now you have to build your own utilities using the rich SOAP or REST APIs that are available in JasperReports Server. The easiest way I found to do this using is using JaspersoftETL (Talend) and a component called <a href="http://www.talendforge.org/exchange/tos/extension_view.php?eid=253">tJasperServerExec</a><br />
<br />
The custom component calls the API of Jaspersoft to handle the report execution end. There are more components like this coming one day soon that let you use the scheduler built into Jaspersoft, for this example we use the mail sending capabilities of Talend.<br />
<br />
What the video doesn't cover is scheduling of this job. The community editions of ETL rely on you to schedule jobs yourself using cron or the Windows scheduler. The commercial editions allow you to publish the job to the Administration Server which have the ability to schedule from a web UI.<br />
<br />
So here's how it's done:<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/G0HuGxSROI4?feature=player_embedded' frameborder='0'></iframe></div>
<br /></div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com2tag:blogger.com,1999:blog-6021256169505852427.post-38341658613679928142012-12-07T04:26:00.002-08:002013-09-13T02:55:10.678-07:00MongoDB Aggregation Queries Basics on MongoLab<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: left;">
What is MongoLab?</h3>
<div>
MongoLab is a hosted <a href="http://mongodb.org/">MonoDB</a> service that offers one free database to developers wanting to learn the technology. It's a very elegant way to start learning or developing a small application. I started using this earlier in the year and had created a couple of collections on it. </div>
<h3 style="text-align: left;">
What is the MongoDB Aggregation Framework?</h3>
<div>
Traditionally in SQL databases you have the concept of a GROUP BY and an aggregation function like average or sum. In MongoDB, aggregation wasn't ever as easy, in order to do any aggregation you'd have to write your own complicated MapReduce jobs. Along comes MongoDB <a href="http://docs.mongodb.org/manual/release-notes/2.2/">version 2.2</a>, which includes the aggregation framework.<br />
<br />
So consider this SQL statement:<br />
<pre>SELECT <span style="background-color: orange;">sum(duration)</span> <span style="background-color: yellow;">FROM calls</span> <span style="background-color: cyan;">GROUP BY disposition</span></pre>
<br />
And here it is translated to MongoDB, I've attempted to color code the translations:</div>
<pre style="border: 1px dashed rgb(153, 153, 153); color: black; font-family: 'Andale Mono', 'Lucida Console', Monaco, fixed, monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><code><span style="background-color: #eeeeee;">
}
runCommand: {
</span><span style="background-color: yellow;">aggregate : 'calls',</span><span style="background-color: #eeeeee;">
pipeline : [
{
$group : {
</span><span style="background-color: cyan;"> _id : '$disposition',</span><span style="background-color: #eeeeee;">
durations: {
</span><span style="background-color: orange;"> $sum : '$duration'</span><span style="background-color: #eeeeee;">
}
}
}
]
}
}
</span></code></pre>
<div>
<br />
Easy enough! There's much more to this of course. Here's a great reference: <a href="http://docs.mongodb.org/manual/reference/aggregation/">http://docs.mongodb.org/manual/reference/aggregation/</a><br />
<br />
<h3 style="text-align: left;">
MongoLabs and Aggregation Framework</h3>
</div>
<div>
So to use this on MongoLabs you'll have to start a new database, old ones are on an older version, according to this <a href="https://support.mongolab.com/entries/20164341-what-version-of-mongodb-does-mongolab-use">helpful article</a></div>
<div>
<br /></div>
<div>
<span style="background-color: white; color: #333333; font-family: 'Century Gothic', 'Heiti TC', sans-serif; font-size: 12px;">All free/starter databases created after Nov 30, 2012 will be running 2.2.x will have </span>Aggregation Framework Support</div>
<div>
<br /></div>
<h3 style="text-align: left;">
Why Do I care? </h3>
<div>
Well I had some mongoDB Jaspersoft samples that would bring in an entire un-aggregated dataset into memory and then use the JasperReports library to do aggregations in-memory. This is fine and dandy if you're dealing with tens of thousands of records. When you move into millions of records this becomes a bit harry and I'd rather my database do the work!</div>
<div>
<br /></div>
<div>
So using my example query from above, I was able to create this very cool looking report in iReport:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvLKziQBymBuSFPC3pJiyUMczrOfP9W23N19HkR56REbtR_gNgGQUxiCTYCLWEzQyMjrvibHgpGM2uqKrI1wTl6w4rNQKqzIRwoYSgFPAPTaKTw6xK5KIfQYcms7n23lUH7Y0ZOAJj0pU/s1600/Screen+Shot+2012-12-07+at+11.35.15+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="334" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvLKziQBymBuSFPC3pJiyUMczrOfP9W23N19HkR56REbtR_gNgGQUxiCTYCLWEzQyMjrvibHgpGM2uqKrI1wTl6w4rNQKqzIRwoYSgFPAPTaKTw6xK5KIfQYcms7n23lUH7Y0ZOAJj0pU/s640/Screen+Shot+2012-12-07+at+11.35.15+AM.png" width="640" /></a></div>
<div>
<br /></div>
<div>
</div>
<div>
<br />
<h3 style="text-align: left;">
Where can I learn more about Reporting and Analytics on MongoDB?</h3>
</div>
<div>
First place to look is the Jaspersoft community wiki: <a href="http://community.jaspersoft.com/wiki/mongodb">http://community.jaspersoft.com/wiki/mongodb</a></div>
<div>
My colleague, Matt Dahlman has some excellent examples on his blog: <a href="http://mdahlman.wordpress.com/tag/mongodb/">http://mdahlman.wordpress.com/tag/mongodb/</a></div>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com1tag:blogger.com,1999:blog-6021256169505852427.post-14204883974138307082012-06-22T09:19:00.003-07:002013-09-13T02:55:23.562-07:00Jaspersoft Report Writing Best Practices<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="p1" style="text-align: left;">
<span style="background-color: white;">The following is a set of guidelines for developing reports with iReport and JasperReports Server. It was compiled with the help of <a href="http://blog.frau-klein.org/">Frau Klein</a></span><br />
<br />
<b style="background-color: white;">Best Practice - report resources:</b><br />
<div class="p1" style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" /></a><span style="background-color: white;">Report units contain references to the resources they use: images, styles, sub-reports, queries, input controls - <a href="http://screencast.com/t/i3fTvZbJW">example</a></span></div>
<div class="p1">
<b>Why</b>?</div>
<div class="p1">
Promotes re-usability (nothing is local to report unit). Easy maintenance around moving report units, replacing resources in a mass scale. </div>
<div class="p1">
<br /></div>
<div class="p1">
<b> Best Practice - references:</b></div>
<div class="p1">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" /></a> Never hard code an entire repository path to resource, the image <i>repo:/reports/images/image.png </i>should come in as a reference like <i>repo:image.png</i><b style="background-color: white;"> </b><span style="background-color: white;">- <a href="http://screencast.com/t/NhbY5X1m">example</a></span></div>
<div class="p1">
<b>Why?</b></div>
<div class="p1">
When you use js-export with --uris option it only resolves repository dependencies, it doesn't look for expressions that maybe use an absolute repository path</div>
<div class="p1">
<br /></div>
<div class="p1">
<b>Best Practice - styles:</b></div>
<div class="p1">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" /></a>Do not hard code any style information to JRXML. Develop central style templates (jrtx, jrctx) for all your reports, add as a reference to reports - <a href="http://screencast.com/t/8V91Pg4CxKtH">example</a></div>
<div class="p1">
<b>Why?</b></div>
<div class="p1">
Fast change management, when the company style changes you will edit a couple of resources vs hundreds. <i>Note that conditional styles are not in the jrtx</i></div>
<div class="p1">
<br /></div>
<div class="p1">
<b>Best Practice - logged in user</b> </div>
<div class="p1">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" /></a> Don't log in as superuser, use an organization specific user like jasperadmin - <a href="http://screencast.com/t/aKgb3JcF">example</a></div>
<div class="p1">
<b>Why?</b></div>
<div class="p1">
Complicates repo: paths. If logged in as superuser you must pass in the full uri ex: <i>repo:/organizations/organization_1/reports/drill_down_report </i>where being part of an organization you would just use<i> repo:/reports/drill_down_report</i>. Also, it's good for testing around multi-tenancy to be a "real" user.</div>
<div class="p1">
<br /></div>
<div class="p1">
<b>Best Practice - input controls</b></div>
<div class="p1">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" /></a>Create shared input controls if they apply to more than one report, bring them in as a link - <a href="http://screencast.com/t/Uyshn928xFT9">example</a></div>
<div class="p1">
<b>Why?</b></div>
<div class="p1">
Reduce development time, useful for dashboards. <i>Note: Datasources won't switch if your sub-report switches</i><br />
<i><br /></i>
<b>Best Practice - Use JNDI Datasources </b><i>(contributed by Guillaume AUTIER)</i><br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" style="cursor: move;" /></a><br />
Have a unique entry point to change datasource details (ip:port:credentials)<br />
<b>Why ?</b><br />
When deploying the server in a different location, you would only have to change the datasource definition in one file: <i>context.xml</i> (this is valid in tomcat, see your app server settings for JNDI)<br />
<br />
When using export scripts the datasource will remain untouched, so you can use the same export on multiple servers without having to change the datasource detail after the import</div>
<div class="p1">
<br /></div>
<div class="p1">
You can take advantage of Application Server pooling instead of creating many connections to the database (performance).<br />
<br />
<b>Best Practice – parametrized references with sub-reports </b><i>(contributed by Guillaume AUTIER)</i><br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGy2wiG-LDa_hBt_LyaIpQ8ov5SQIee3wv1SshtdHmGM2nxCX6ObHVFWUxIK9TQH8IzK0XMnxIzxT4Kato5JRnO29HBRp0QkJCfggJb7eaE5hdSf_V00JgGn23YhIAxk6zSFtHJS22Mf0/s1600/lightbulb.png" /></a>Using the previous advice you can also access a report (with subreports) both locally and on the server without touching your expressions. <br />
<b>Why?</b><br />
This is useful when you want to quickly preview in iReport a report taken from the server. We assume that all reports and subreports .jasper files resides in the same local folder.<br />
<br />
<br />
<ul style="text-align: left;">
<li>Create a parameter <i>$P{SUB_REPORTDIR} </i>(string type, use as prompt, default value “repo:”)</li>
<li>Then have the following formula in your subreport expressions</li>
</ul>
<br />
<br />
<code>$P{SUB_REPORTDIR }.equals(“”)?”mySubReport.jasper”: $P{ SUB_REPORTDIR} + ”mySubReport.jrxml”</code><br />
<br />
<br />
<ul style="text-align: left;">
<li>Publish your report on the server as usual : do not create(or link) an input control for the SUB_REPORTDIR parameter</li>
</ul>
<br />
Run your report :<br />
<br />
<ul style="text-align: left;">
<li>In iReport you will be prompted for the SUB_REPORTDIR parameter leave it blank.</li>
<li>On the server : Without an input control for the SUB_REPORTDIR parameter the server will then take the default.</li>
</ul>
<br />
<br />
<i>If you have any other best practices tips, post them in the comments and I'll include into the post, happy reporting!</i></div>
</div>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com1tag:blogger.com,1999:blog-6021256169505852427.post-11261063967641867692012-04-12T10:23:00.001-07:002013-09-13T02:55:34.577-07:00TalendDate<div dir="ltr" style="text-align: left;" trbidi="on">
Not a lot of examples around TalendDate so thought I'd post a couple of quick examples:<br />
<br />
<b>Today's date:</b><br />
<code>TalendDate.getDate("yyyy-MM-dd") </code><br />
<b>Returns:</b><br />
2012-04-12<br />
<br />
<b>Yesterday's date:</b><br />
<code>TalendDate.addDate(TalendDate.getDate("yyyy-MM-dd"),"yyyy-MM-dd",-1,"dd")</code><br />
<b>Returns:</b><br />
2012-04-11<br />
<br />
<b>Tomorrow's date:</b><br />
<code>TalendDate.addDate(TalendDate.getDate("yyyy-MM-dd"),"yyyy-MM-dd",1,"dd")</code><br />
<b>Returns:</b><br />
2012-04-13<br />
<br />
<b>Date format:</b><br />
CC - century<br />
YY - year<br />
MM - month<br />
DD - day of month<br />
hh - hour<br />
mm - minutes<br />
ss - seconds<br />
<br />
<b>Return types:</b><br />
<b>string!</b><br />
<b><br />
</b><br />
<b>Other methods:</b><br />
<br />
<span style="font-family: monospace;"></span><br />
<span style="font-family: monospace;">TalendDate.compareDate(myDate,myDate2,"yyyy-MM-dd")</span><br />
<span style="font-family: monospace;">TalendDate.diffDate(myDate,myDate2,"MM")</span><br />
<span style="font-family: monospace;">TalendDate.formatDate("yyyy-MM-dd HH:mm:ss",myDate)</span><br />
<span style="font-family: monospace;">TalendDate.setDate(mydate,newValue,"MM")</span><br />
<span style="font-family: monospace;">TalendDate.getFirstDayOfMonth(mydate)</span><br />
<span style="font-family: monospace;">TalendDate.getLastDayOfMonth(mydate)</span><br />
<span style="font-family: monospace;">TalendDate.getRandomDate("2007-01-01","2008-12-31")</span><br />
<span style="font-family: monospace;">TalendDate.parseDate("yyyy-MM-dd HH:mm:ss","")</span><br />
<br />
<br />
<div class="p1">
<code></code></div>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com1tag:blogger.com,1999:blog-6021256169505852427.post-61246745957226398052011-12-15T17:57:00.000-08:002013-09-13T02:55:47.544-07:00Flush Jaspersoft OLAP Cache from Talend/JasperETL<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://www.jaspersoft.com/jasperetl">JaspersoftETL</a> is a Java code generation tool designed to load your operational data into a <a href="http://en.wikipedia.org/wiki/Data_warehouse">data warehouse</a> and get it ready for reporting and analysis. <a href="http://www.jaspersoft.com/jaspersoft-olap">Jaspersoft OLAP</a> takes advantage of aggregate tables and an in-memory cache to speed up queries. If you're running a nightly job to load the DW, it's a good idea to clear the OLAP cache at that time, otherwise your users will not take advantage of new data.<br />
<br />
There are several ways to clear the OLAP cache:<br />
<ul style="text-align: left;">
<li>Login with <span class="s2">superuser </span>privileges and display the OLAP Settings page.</li>
<li>In the OLAP Settings panel, click <span class="s3"><b>Flush OLAP Cache</b></span>. </li>
<li><span class="s1"><span class="Apple-tab-span"> </span></span>Using utilities such as cURL and wget, send an HTTP request to this URL: <code>http://server:8080/<server><port><context-path>olap/flush.html?j_username=superuser&j_password=<superuser_password></superuser_password></context-path></port></server></code></li>
<li>Restart the application server instance. </li>
<li>Call the Java API.</li>
</ul>
<div>
I've created a small component that allows the clearing of cache via the URL method from a JasperETL/Talend job. Essentially it will prompt you for a username and password (and note that only superuser can clear the cache by default) and a server URL. The URL should point to something like <code>http://server:8080/jasperserver-pro/olap/flush.html</code></div>
<div>
<br /></div>
<div>
You may find the tJasperOLAPCache component on <a href="http://www.talendforge.org/exchange/index.php?eid=443&product=tos&action=view&nav=1,1,1">talendforge.org</a></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIuO5qhYo6Sik5nKOI5GyM27A2OkWf2FnB8HzWdvPRaZMyQAyzNnYm23QWxDyZKd56ePUPQeXcGoiO-lyXvX_67E0j2a5mdTUQCj9Q7P6vxX-f3KXfR9tnl-A2VzCLTaOcODsiT73RWC0/s1600/Screen+shot+2011-12-15+at+5.29.28+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="253" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIuO5qhYo6Sik5nKOI5GyM27A2OkWf2FnB8HzWdvPRaZMyQAyzNnYm23QWxDyZKd56ePUPQeXcGoiO-lyXvX_67E0j2a5mdTUQCj9Q7P6vxX-f3KXfR9tnl-A2VzCLTaOcODsiT73RWC0/s400/Screen+shot+2011-12-15+at+5.29.28+PM.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Another useful thing to do would be to use the <a href="http://www.talendforge.org/exchange/index.php?eid=253&product=tos&action=view&nav=0,1,1">tJasperServerExec</a> plugin to re-populate the cache.</div>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com1tag:blogger.com,1999:blog-6021256169505852427.post-87831653475961923772011-12-01T17:50:00.000-08:002013-09-13T02:55:58.128-07:00Using tGoogleAnalyticsInput in JasperETL/Talend<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
This is a short article on getting Google Analytics into <a href="http://www.jaspersoft.com/jasperetl">JaspersoftETL</a>/<a href="http://www.talend.com/">Talend ETL</a> tools. <a href="http://www.google.com/analytics/">Google Analytics</a> has a nice interface and many great ways of analyzing web traffic, it'd be hard to beat the work Google did there. The purpose of this connector would be to bring the data into your own data warehouse and join it with other intelligence you might have. A great way to display the aggregated data would be with a BI tool like <a href="http://jaspersoft.com/">Jaspersoft</a>.<br />
<br />
I'm writing because the <a href="http://blog.smile.fr/Recuperer-ses-donnees-de-trafic-de-GoogleAnalytics-avec-Talend-et-Smile">help for the tGoogleAnalyticsInput is in French</a> and incomplete.<br />
<br />
So let's get started. You'll need JasperETL 4.1.2 or older (commercial editions <a href="http://jaspersoft.com/">here</a>, open source <a href="http://jasperforge.org/">here</a>).<br />
<br />
Installing the plugin:<br />
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAXqxRkeoPmNNDhCsv_OtgvY2Jumg3gxNzqKtqJdZNRu0DmzcMrCRLnlWjlJ0MSgdklLIpBW4bhvC5jfd_Kyb7fdnIMsnHV7h8crgB5ScsJUx7EXk_3lquGu9azqIqu3ZD5Td5i6udWBo/s1600/install.png" imageanchor="1" style="clear: left; margin-bottom: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAXqxRkeoPmNNDhCsv_OtgvY2Jumg3gxNzqKtqJdZNRu0DmzcMrCRLnlWjlJ0MSgdklLIpBW4bhvC5jfd_Kyb7fdnIMsnHV7h8crgB5ScsJUx7EXk_3lquGu9azqIqu3ZD5Td5i6udWBo/s1600/install.png" /></a></div>
<div style="text-align: left;">
Create a new Job Design and drag the tGoogleAnalyticsInput component from the Palette (under Internet):</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUy1NvLTShPrHEIztGkhchBwtzEhPO9InvBGRawE6WKPuAKrR9Qt2uHELLtOK4tEy5ndYErxMVK_J3SgGWY07TC-uBczEvh473wzPnevcvTY4irwkzn2YDzmXqLETiXHyzkvLWgqHHJMA/s1600/component.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUy1NvLTShPrHEIztGkhchBwtzEhPO9InvBGRawE6WKPuAKrR9Qt2uHELLtOK4tEy5ndYErxMVK_J3SgGWY07TC-uBczEvh473wzPnevcvTY4irwkzn2YDzmXqLETiXHyzkvLWgqHHJMA/s1600/component.png" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Configure the component. First start with finding some fields you'd like to use from the Google API: http://code.google.com/apis/analytics/docs/gdata/dimsmets/dimsmets.html. Note that you need to have some metrics (can't be all dimensions, vice versa).</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Here's an example schema (you'll have to put in the fields yourself):</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij6IxOKrPf_JYvu_Mjvs41duXuOm9FgS3n0B8uJY28VdYcImSXHrdtK6vNbBdDhrdY2XUu6Rr8mzbsmaMrIxs4tdFDFRtrLskGbfZUpaDlIuzkf5oqwRoL9U_bDqFUj3_5ne7Yky5Bo5k/s1600/schema.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij6IxOKrPf_JYvu_Mjvs41duXuOm9FgS3n0B8uJY28VdYcImSXHrdtK6vNbBdDhrdY2XUu6Rr8mzbsmaMrIxs4tdFDFRtrLskGbfZUpaDlIuzkf5oqwRoL9U_bDqFUj3_5ne7Yky5Bo5k/s1600/schema.png" /></a></div>
</div>
<div style="text-align: left;">
Now configure the component. In order to get your Profile ID (note it doesn't start with UA) read the cryptic Google Documentation about it. I got mine by examining the URL while logged into GA and looking at the &id= parameter.<br />
<br />
The "value" column comes directly from the Google API and will also have to be filled in. I've also found that I HAVE to filter on period. You can do 1980-2020 if you want to include everything. </div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0EseSckd3tK2A57O3NAVY5fVZRxoPxpmSJZWjOAMgR56Xe33m7m6tE553v3qorOtfpmNZk5vgg5EO4YO0URQIEuYsG6TF8ARJif-D4if5Z66VXPCy7Uj4cLFNi8sDq_7flKh6FDGyDJ0/s1600/component_configure.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0EseSckd3tK2A57O3NAVY5fVZRxoPxpmSJZWjOAMgR56Xe33m7m6tE553v3qorOtfpmNZk5vgg5EO4YO0URQIEuYsG6TF8ARJif-D4if5Z66VXPCy7Uj4cLFNi8sDq_7flKh6FDGyDJ0/s1600/component_configure.png" /></a></div>
<div style="text-align: left;">
<br /></div>
Now you can run the the job and see your visitors:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdpl3SnaV_Mo9IJh4SI2L0YmP6NRh0fCVaCIxUyWUbfNUAXLZdZabLLyh1kx7MeEX8eQyk2PbVjjqNCIA-XOpuD5baAOs7Rylpis1rljVMkoNZ7PsXbSP435bkJ_2MBOEcNO_A5fMgmB0/s1600/output.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdpl3SnaV_Mo9IJh4SI2L0YmP6NRh0fCVaCIxUyWUbfNUAXLZdZabLLyh1kx7MeEX8eQyk2PbVjjqNCIA-XOpuD5baAOs7Rylpis1rljVMkoNZ7PsXbSP435bkJ_2MBOEcNO_A5fMgmB0/s1600/output.png" /></a></div>
<br />
Funny enough. If you're reading this right now then you will eventually flow through the GA database and the very job I'm displaying above ;)</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com5tag:blogger.com,1999:blog-6021256169505852427.post-31068791673410627892011-08-01T12:14:00.000-07:002013-09-13T02:56:11.214-07:00Icinga Reporting with JasperReports Server - Part 2: Usage + Basic Report Development<div dir="ltr" style="text-align: left;" trbidi="on">
Short tutorial, shows how to schedule a report, how to change the way you are prompted for parameters and some basic editing of reports within iReport<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/I3_wKmuWmsY/0.jpg" height="266" width="320"><param name="movie" value="http://www.youtube.com/v/I3_wKmuWmsY?f=user_uploads&c=google-webdrive-0&app=youtube_gdata" /><param name="bgcolor" value="#FFFFFF" /><embed width="600" height="600" src="http://www.youtube.com/v/I3_wKmuWmsY?f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div>
</div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-41260775817029836302011-08-01T11:30:00.000-07:002013-09-13T02:56:20.952-07:00Icinga Reporting with JasperReports Server - Part 1: Installation<div dir="ltr" style="text-align: left;" trbidi="on">
A quick overview of how to install and configure the reporting add-on packages for Icinga (a Nagios Monitoring Fork), written for JasperReports Server.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/SHLsTx3q6KY?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<span id="goog_1872711880"></span><span id="goog_1872711881"></span></div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-52720218052571645502011-04-18T11:59:00.000-07:002013-09-13T02:56:30.282-07:00Backing Up JasperServer in Linux<div dir="ltr" style="text-align: left;" trbidi="on">
This is a sysadmin how-to for backing up JasperReports Server (formerly JasperServer) and it's database (MySQL only for now).<br />
<br />
I wrote a fairly robust script to achieve this task. It will read the database configuration from your default_master.properties file, work with several versions of JasperServer, delete old backups, write to the syslog and export the repository contents to file.<br />
<br />
First drop my backup script into your <i><js-install>\buildomatic</js-install></i> folder if you're using 4.0 and abovie, 3.7 and below put in your <i>/scripts</i> folder.<br />
<br />
<b>Once the script is in place edit the variables at the top of the script:</b><br />
<br />
This is to signify that you're using JasperServer 3.7.1 and below:<br />
<i>IS_PRE_4=false</i><br />
<br />
This is a name for the product:<br />
<i>PRODUCT_NAME="JRS-4.0-Pro"</i><br />
<br />
This is where backups will be stored:<br />
<i>BACKUP_DEST="/home/ernesto/Dropbox/Backups"</i><br />
<br />
This will delete old backups if set to true<br />
<i>DELETE_OLD=true</i><br />
<br />
How many days to keep old backups<br />
<i>DAYS=120</i><br />
<br />
Now add it to your crontab. I will do it once a week in a lazy manner so will add the script to vim /etc/cron.weekly/js-backups<br />
<br />
<code>#!/bin/bash</code><br />
<code>/usr/local/jaspersoft/jasperserver4/buildomatic/js-backup.sh</code><br />
<br />
<br />
The script will log to /var/log/messages:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4CdKGxZHmKidXfbdepasMI9mkRJjwkv7FPUJhZZ_d5oxzWO6V7j6_vIU99cEhpwyWuf30FHdbdxpews9YnfftNDlPVUtj5IhF405coelxvbA551zSekyfBxqRCtQEgDkNSd21unGMhok/s1600/log.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="105" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4CdKGxZHmKidXfbdepasMI9mkRJjwkv7FPUJhZZ_d5oxzWO6V7j6_vIU99cEhpwyWuf30FHdbdxpews9YnfftNDlPVUtj5IhF405coelxvbA551zSekyfBxqRCtQEgDkNSd21unGMhok/s640/log.png" width="640" /></a></div>
<br />
<br />
The output files will look like this, I chose to stick them in my <a href="http://dropbox.com/">Dropbox</a> folder for off-site backups:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpsJDj9msQ4LKThXq6DufgnVND2bqFJfj1MJDWymvK1ur9C1LHXB2XVlqF1M-RZUp6vJwfMzfUxOu4Upcbp7vKkI2QbMQjKDrLlAroSGLfSSOpaMU7nSFW3kpcP6-Sb82oBwS0E18EQJQ/s1600/backup_dir.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpsJDj9msQ4LKThXq6DufgnVND2bqFJfj1MJDWymvK1ur9C1LHXB2XVlqF1M-RZUp6vJwfMzfUxOu4Upcbp7vKkI2QbMQjKDrLlAroSGLfSSOpaMU7nSFW3kpcP6-Sb82oBwS0E18EQJQ/s400/backup_dir.png" width="400" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpsJDj9msQ4LKThXq6DufgnVND2bqFJfj1MJDWymvK1ur9C1LHXB2XVlqF1M-RZUp6vJwfMzfUxOu4Upcbp7vKkI2QbMQjKDrLlAroSGLfSSOpaMU7nSFW3kpcP6-Sb82oBwS0E18EQJQ/s1600/backup_dir.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br />
</a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpsJDj9msQ4LKThXq6DufgnVND2bqFJfj1MJDWymvK1ur9C1LHXB2XVlqF1M-RZUp6vJwfMzfUxOu4Upcbp7vKkI2QbMQjKDrLlAroSGLfSSOpaMU7nSFW3kpcP6-Sb82oBwS0E18EQJQ/s1600/backup_dir.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br />
</a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpsJDj9msQ4LKThXq6DufgnVND2bqFJfj1MJDWymvK1ur9C1LHXB2XVlqF1M-RZUp6vJwfMzfUxOu4Upcbp7vKkI2QbMQjKDrLlAroSGLfSSOpaMU7nSFW3kpcP6-Sb82oBwS0E18EQJQ/s1600/backup_dir.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br />
</a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
You can download the script from this<a href="http://jasperforge.org/plugins/espforum/view.php?group_id=112&forumid=102&topicid=86513"> jasperforge forum post</a> or view it below:<br />
<br />
<div class="snipplr_embed" id="snipplr_embed_52290">
<a href="http://snipplr.com/view/52290/jasperserver-backup-script/">Code snippet - JasperServer Backup Script</a> on Snipplr</div>
<script src="http://snipplr.com/js/embed.js" type="text/javascript">
</script><script src="http://snipplr.com/json/52290" type="text/javascript">
</script></div>
Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com1tag:blogger.com,1999:blog-6021256169505852427.post-34540145185491323262011-04-01T00:28:00.000-07:002011-04-01T00:28:14.710-07:00Music Player Daemon w/ lame Support in DebianTrying to get my old Blackberry to stream from my <a href="http://www.musicpd.org/">mpd</a> server, it turns out the built in http stream output is not compiled with lame encoder (mp3) support. Here are some instructions on building the source package with the lame encoder options compiled in for Debian Squeeze (6.0)<br />
<br />
(as root)<br />
Install libmp3lame0 and lame from http://debian-multimedia.org/<br />
<blockquote>cd /tmp/</blockquote>Get the build dependencies for mpd:<br />
<blockquote>apt-get build-dep mpd</blockquote>Get the mpd source package:<br />
<blockquote>apt-get source mpd </blockquote><blockquote>cd mpd-0.15.12 (your version may differ)</blockquote>Edit the file <i>debian/rules</i> and remove <i>--disable-lame</i><br />
Build the binary:<br />
<blockquote>./debian/rules binary</blockquote>Build the .deb package:<br />
<br />
<blockquote>cd .. </blockquote><blockquote>dpkg-source -b mpd-0.15.12</blockquote>Install the package:<br />
<br />
<blockquote>dpkg -i mpd_0.15.12-1.1_amd64.deb</blockquote>Lock the package in place so that updates won't blow away your change:<br />
<blockquote>echo mpd hold | dpkg --set-selections</blockquote><div>That's it! Then you can configure an audio output in <i>/etc/mpd.conf:</i></div><div><i><div><code></code></div><div><code>audio_output {</code></div><div><code> type "httpd"</code></div><div><code> name "My HTTP Stream"</code></div><div><code> encoder "lame"</code></div><div><code> port "8000"</code></div><div><code> quality "5.0" </code></div><div><code><span class="Apple-style-span" style="font-style: normal;"><i><div style="display: inline !important;"> always_on "yes" #This option doesn't seem to work for me :(</div></i></span></code></div><div><code><span class="Apple-style-span" style="font-style: normal;"><i><div style="display: inline !important;">}</div></i></span></code></div><div></div></i></div>Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com2tag:blogger.com,1999:blog-6021256169505852427.post-7850760236176373782011-03-24T16:58:00.000-07:002011-03-24T16:58:43.148-07:00ThinkPad Overheating Issue Fix<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSHlQR8C-JcMn7MpxKEfMJgNfmqybeU9EcI5zNzIePRiG_8r9klWSMr5R9aH_IsQNBN4B7HcpUvbyoPFXRTcFDzJT30XF2HhXURYiJJo2GY5jInXpDqiQo78nqEPhL9eArQv6pl0pP_GY/s1600/IMAG0010.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSHlQR8C-JcMn7MpxKEfMJgNfmqybeU9EcI5zNzIePRiG_8r9klWSMr5R9aH_IsQNBN4B7HcpUvbyoPFXRTcFDzJT30XF2HhXURYiJJo2GY5jInXpDqiQo78nqEPhL9eArQv6pl0pP_GY/s320/IMAG0010.jpg" width="191" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">She's pretty hot!</td></tr>
</tbody></table>On my ThinkPad T410s fans were disabled by default after installing Debian 6 (Squeeze).<br />
<br />
I would randomly be working on CPU intensive stuff and the computer would shut down with:<br />
<blockquote><span class="Apple-style-span" style="font-family: Sans, Arial, Helvetica, sans-serif; font-size: 13px;"><pre style="background-color: white; border-bottom-color: rgb(85, 85, 85); border-bottom-style: solid; border-bottom-width: 0px; border-color: initial; border-left-color: rgb(85, 85, 85); border-left-style: solid; border-left-width: 0px; border-right-color: rgb(85, 85, 85); border-right-style: solid; border-right-width: 0px; border-style: initial; border-top-color: rgb(85, 85, 85); border-top-style: solid; border-top-width: 0px; color: black; font-family: 'bitstream vera sans mono', monaco, monospace; font-size: 1em; letter-spacing: 0px; line-height: 1.4em; margin-bottom: 0px; margin-left: 10px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;">Critical temperature reached (128 C)</pre></span></blockquote> To fix this I created the file /etc/modprobe.d/thinkpad_acpi.conf and added options thinkpad_acpi fan_control=1 to it. I've submitted <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=619549">a bug against the acpi package in Debian.</a><br />
<br />
I got help from <a href="http://www.thinkwiki.org/wiki/Thinkpad-acpi">ThinkWiki</a> on this.<br />
<br />
<br />
<br />
<div><br />
</div>Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com1tag:blogger.com,1999:blog-6021256169505852427.post-66515269825394475502011-03-09T14:08:00.000-08:002011-03-09T14:08:39.656-08:00File Management Tip for Nautilus/Gnome in Debian<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgd5SxVafpDAwghiMOsIc2yAzgSReIeCcXqQNkOvqmj-c0TUdDzTdfTsZcGjl3FcPSKzhTRWw7v0XfrNf3vy0Ya0BSbVcNfhLO2vustVu4brWkyc9uxXSUojaGzbl8qAGST09AbiiTHtnA/s1600/Screenshot-lib+-+File+Browser.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgd5SxVafpDAwghiMOsIc2yAzgSReIeCcXqQNkOvqmj-c0TUdDzTdfTsZcGjl3FcPSKzhTRWw7v0XfrNf3vy0Ya0BSbVcNfhLO2vustVu4brWkyc9uxXSUojaGzbl8qAGST09AbiiTHtnA/s1600/Screenshot-lib+-+File+Browser.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">nautilus-open-terminal in action</td></tr>
</tbody></table>Often I find myself browsing to a directory in Nautilus and then realizing that I'd rather be in terminal. Other times I want to be in Nautilus when I'm in the terminal. There's certain things the command line is better at than the file browser and vice versa. Switching between the two is fairly easy in Gnome/Nautilus.<br />
<br />
<b>Going from Nautilus to Terminal:</b><br />
You'll need the nautilus-open-terminal package (<code> sudo apt-get install nautilus-open-terminal; killall -HUP nautilus </code>). Then in any Nautilus, right click on white space and select "Open in Terminal"<br />
<br />
<b>Going from Terminal to Nautilus:</b><br />
This one is simpler. I watched a guy at work do this on his Mac and got jealous. You can go from Terminal to Nautilus opened in your current directory by typing: <i>nautilus .</i><br />
<br />
<blockquote><br />
ernesto@debian-t410s:~$ nautilus .<br />
</blockquote>Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-62136937090790349422011-01-16T20:15:00.000-08:002011-01-16T20:15:27.314-08:00Linux zd1211 Wireless Speed FixFor some reason my Belkin 54Mbps USB wifi adapter operates at 1 Mb/s under Linux.<br />
It's detected as the following by lsusb:<br />
<code></code><br />
Bus 001 Device 002: ID 050d:705c Belkin Components F5D7050 Wireless G Adapter v4000 [Zydas ZD1211B]<br />
<br />
If I do a <code>iwconfig wlan0 rate 54M</code> the card starts working at 54Mb/s. To fix this more permanently I added the following to my<i> /etc/network/interfaces</i> file:<br />
<code></code><br />
auto wlan0<br />
iface wlan0 inet dhcp<br />
wpa-ssid 56k<br />
wpa-psk secret<br />
<i>wireless-rate 54M</i>Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-61547290216798338952010-12-07T13:26:00.000-08:002010-12-07T13:26:11.434-08:00Solaris on VirtualboxTrying out <a href="http://en.wikipedia.org/wiki/Solaris">Solaris </a>on <a href="http://www.virtualbox.org/">Virtualbox</a> to troubleshoot an issue at <a href="http://www.jaspersoft.com/">work</a> with no luck booting. The VM would hang after the grub prompt.<br />
<br />
Disabling USB support and sound support did it for me.Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-36447099151118517422010-11-05T14:02:00.000-07:002010-11-05T14:02:38.164-07:00Cron Jobs in the Green Data CenterYesterday I wrote a script to monitor a UPS (running <a href="http://www.apcupsd.com/">apcupdsd</a>) server using <a href="http://nagios.org/">Nagios</a>. Today I'm looking at some of the results of the performance data. It seems that the hourly cron jobs on my server actually cause some power usage. I'll have to think over how often it's actually necessary to run those scripts given it's sucking a noticable bit of juice. Check out the graph (produced by <a href="http://docs.pnp4nagios.org/pnp-0.4/start">PNP for Nagios</a>)<div><br />
</div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPjqdxqfuRgtpjfvt2Rv2sL5cH5vnc_FdHW4jyYtZwZzye0ehZCDm7HJyBAagHO-wJ_FDHt4MfCQKVZOMc_xLY6yXES2NZnTZMzDuu5sWD5E8jxWq99zCFDeVbceZ-iyFqnGfatFuAnVQ/s1600/index.php+(1).png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="181" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPjqdxqfuRgtpjfvt2Rv2sL5cH5vnc_FdHW4jyYtZwZzye0ehZCDm7HJyBAagHO-wJ_FDHt4MfCQKVZOMc_xLY6yXES2NZnTZMzDuu5sWD5E8jxWq99zCFDeVbceZ-iyFqnGfatFuAnVQ/s640/index.php+(1).png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">4 hour Load Percentage of our new <a href="http://www.apc.com/products/resource/include/techspec_index.cfm?base_sku=SUA2200R2X106">APC 2200</a> @ <a href="http://etszone.com/">work</a></td></tr>
</tbody></table><div><br />
</div>Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-66761535102084534012010-10-07T11:38:00.000-07:002010-10-07T11:38:34.935-07:00SSH Client Keep AlivesAt my new office in the <a href="http://en.wikipedia.org/wiki/Shell_Building_(San_Francisco)">Shell Building</a> the main router is a <a href="http://www.sonicwall.com/">SonicWal</a>l which has a setting to drop idle TCP connections after 15 minutes. This is annoying since I usually have several going at once. I don't have access to the router so I had to work around the issue.<br />
<br />
The poor man's solution is to run the "top" command on your idle terminals so that packets come through. Not elegant and too manual for me.<br />
<br />
The rich man's solution is to change the server's keep alive settings (TCPKeepAlive yes). I call this the rich man's solution because it means every one bends over for you.<br />
<br />
The middle class man's solution is to add some directives to the default SSH client options on your local machine. Here's how:<br />
<br />
Edit ( as root) <i><b>/etc/ssh/ssh_config</b> </i>and add the following lines:<br />
<br />
<code>TCPKeepAlive yes</code><br />
<code>ServerAliveInterval 60</code><br />
<span class="Apple-style-span" style="font-family: monospace; white-space: pre;">ServerAliveCountMax 5</span><br />
<br />
<div><br />
</div><div>This means that every 60 seconds your machine will send a keep alive to the server if there is no other activity. If the physical connection gets dropped then every 60 seconds the keep alive will be sent until it's done that 5 times, finally giving up (in 5 minutes).</div><div><br />
</div><div>From the <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=ssh_config">man</a> page:</div><b><div><b><br />
</b></div><span class="Apple-style-span" style="font-size: small;">TCPKeepAlive</span></b><div><b><span class="Apple-style-span" style="font-size: small;"></span></b><span class="Apple-style-span" style="font-size: small;">Specifies whether the system should send TCP keepalive messages to the other side. If they are sent, death of the connection or crash of one of the machines will be properly noticed. However, this means that connections will die if the route is down temporarily, and some people find it annoying. The default is ``yes'' (to send TCP keepalive messages), and the client will notice if the network goes down or the remote host dies. This is important in scripts, and many users want it too. To disable TCP keepalive messages, the value should be set to ``no''.<br />
</span><b><span class="Apple-style-span" style="font-size: small;">ServerAliveCountMax</span></b></div><div><b><span class="Apple-style-span" style="font-size: small;"></span></b><span class="Apple-style-span" style="font-size: small;"> Sets the number of server alive messages (see below) which may be sent without ssh(1) receiving any messages back from the server. If this threshold is reached while server alive messages are being sent, ssh will disconnect from the server, terminating the session. It is important to note that the use of server alive messages is very different from TCPKeepAlive (below). The server alive messages are sent through the encrypted channel and therefore will not be spoofable. The TCP keepalive option enabled by TCPKeepAlive is spoofable. The server alive mechanism is valuable when the client or server depend on knowing when a connection has become inactive. The default value is 3. If, for example, ServerAliveInterval (see below) is set to 15 and ServerAliveCountMax is left at the default, if the server becomes unresponsive, ssh will disconnect after approximately 45 seconds. This option applies to protocol version 2 only.<br />
</span><b><span class="Apple-style-span" style="font-size: small;">ServerAliveInterval</span></b></div><div><b><span class="Apple-style-span" style="font-size: small;"></span></b><span class="Apple-style-span" style="font-size: small;"> Sets a timeout interval in seconds after which if no data has been received from the server, ssh(1) will send a message through the encrypted channel to request a response from the server. The default is 0, indicating that these messages will not be sent to the server. This option applies to protocol version 2 only.</span><br />
</div>Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-17661535317898365302010-09-17T11:01:00.000-07:002010-09-17T19:53:38.000-07:00Adobe Flash is back for amd64!Finally the 64 bit edition of Flash is somewhat available again! It's called Flash "Square". I tried it last night on my dual core Atom HTPC running Debian Squeeze. It was amazing. <i>Full screen HD YouTube videos play smooth as butter</i>. I've never seen this with Flash in Linux! Amazing!<br />
<br />
By the way I still hate Flash but you gotta live with it so...<br />
<br />
<i>See my edits in the Debian wiki on how to install on a 64 bit platform:</i><br />
<a href="http://wiki.debian.org/FlashPlayer#Debian5.0.27Lenny.27and.27Squeeze.27amd64"><b>http://wiki.debian.org/FlashPlayer#Debian5.0.27Lenny.27and.27Squeeze.27amd64</b></a>Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-58394347483496440612010-09-17T10:00:00.000-07:002010-09-17T10:00:45.829-07:00d.o.t.s in your Gmail address demystifiedThis is probably not a surpsrise to anyone but me but I still want to share.<br />
<div><br />
</div><div>According to Google:</div><div><ul></ul><ul><li><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; font-family: Arial, Helvetica, sans-serif; font-size: 13px;">homerjsimpson@gmail.com = hom.er.j.sim.ps.on@gmail.com</span></li>
<li><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; font-family: Arial, Helvetica, sans-serif; font-size: 13px;">homerjsimpson@gmail.com = HOMERJSIMPSON@gmail.com</span></li>
<li><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; font-family: Arial, Helvetica, sans-serif; font-size: 13px;"></span><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; font-family: Arial, Helvetica, sans-serif; font-size: 13px;">homerjsimpson@gmail.com = Homer.J.Simpson@gmail.com</span></li>
</ul></div><div>Sweet! Now to make it even harder to contact me just e-mail me at e(dot)r(dot)n(dot)e(dot)s(dot)t(dot)o(dot)o(dot)n(dot)g(dot)a(dot)r(dot)o AT gmail(dot)com</div><div><br />
</div><div><a href="http://mail.google.com/support/bin/answer.py?hl=en&ctx=mail&answer=10313#">http://mail.google.com/support/bin/answer.py?hl=en&ctx=mail&answer=10313#</a></div>Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0tag:blogger.com,1999:blog-6021256169505852427.post-57508896426164111542010-08-30T20:43:00.000-07:002010-08-31T11:09:34.923-07:001,741 times faster than dial up!I am checking out the grand new offices for the <a href="http://fiz.stanford.edu:8081/display/ramcloud/Home">RAMCloud project</a> at <a href="http://stanford.edu/">Stanford</a>. The internet here is 1,741 times faster than 56k dial-up! Ironically storage speeds have not grown at the same rate as Internet speeds, that's what RAMCloud hopes to solve (in the data center).<br />
<br />
This is insane bandwidth. Oh and every PC here gets a public IP! I'm in Internet heaven.Ernesto Ongarohttp://www.blogger.com/profile/04102517836579717069noreply@blogger.com0