Posts

Showing posts with the label Batch Processing

S) Normalise scenario --> Input: 1,a,b,c,2,d,e,f,g

 Input: 1,a,b,c,2,d,e,f,g   Output: field1 field2 1       a 1       b 1       c 2       d 2       e 2       f 2       g     input---->normalize--->output   type temporary_type =  record   decimal("") v_col1;   decimal("") v_col2; end; out::length(in) = begin   out :: length_of(string_split(in.col, ",")); end; temp::normalize(temp, in, index) = begin   temp.v_col1 :: if (string_is_numeric(string_split(in.col, ",")[index])==1)                      string_split(in.col, ",")[index] else temp.v_col1;   temp.v_col2 :: string_split(in.col, ",")[index]; end; temp::initialize(in) = begin   temp.v_col1 :: 0;   temp.v_col2 :: 0; end; out::finalize(temp, in) = begin   out.col1 :: temp.v_col1;   out.col2 :: temp.v_col2; end; ...

S) I have 100 records in input file in output i want 1 million records. What will be best approach? #scenario

 Use normalize with length as 10,000 fixed number. Graph will be having below components to produce output: Input file-> normalize -> output file

S) How to add trailer record at every Third record of a file having 10 records? #scenario

In output index of reformat use logic if next in sequence % 3 == 0 then output add trailer from in1.    you can use normalize generate length as 2 when next in sequence %3 ==0.