Informatica Q&A › Processing large files with Informatica DataTransformation
<Guest>
Dear community ,
I’m working with Informatica B2B DataTransformation and the unstructured source file is very large (can get to more than 750 MB),How Can I tackle this within the tool ?
<ABlinkconsulting>
Informatica Data Transformation offers a Streamer functionality, geared exactly for the use case you are describing above – processing unlimited size of files. It does that by chunking the input files into logical chunks, and processing each chunk separately. You can use Streamer option in data transformation to break down dynamically the file to small chunks with header and body according to file structure . Each streamer part (header_segment, repeating_segment and fotter_segment can be extracted using a dedicated parser which you can call for each part. See attached is a screenshot of defining a steamer in DT.
