Array to write into files by splits
hi,
have array contains around five-hundred-thousands values (sorted numbers). want value array , writing values splitting multiple files. conditions are:
- want write 50000 array rows each file sequentially giving file numbers x_1, x_2 ... x_10 etc.
- 1 file can contain "multiple id" comes sequentially
- but, "multiple files" cannot contain "same id" (value)
- if going 50000th line in array see next-few-rows has same values 50000th line, include them writing same file, cannot take same id file
- so, write (50000 values) or (50000 values + continuation of same value) 1 file
part of big process, able build until array, facing trouble write values splitting files following logic. can help?
sample array ($a):
9001
9001
9001
9001
9002
9002
9003
9003
9003
9003
9003
9004
9004
9004
9004
9004
9004
9004
9005
9006
9006
9006
have array contains around five-hundred-thousands values (sorted numbers). want value array , writing values splitting multiple files. conditions are:
- want write 50000 array rows each file sequentially giving file numbers x_1, x_2 ... x_10 etc.
- 1 file can contain "multiple id" comes sequentially
- but, "multiple files" cannot contain "same id" (value)
- if going 50000th line in array see next-few-rows has same values 50000th line, include them writing same file, cannot take same id file
- so, write (50000 values) or (50000 values + continuation of same value) 1 file
part of big process, able build until array, facing trouble write values splitting files following logic. can help?
sample array ($a):
9001
9001
9001
9001
9002
9002
9003
9003
9003
9003
9003
9004
9004
9004
9004
9004
9004
9004
9005
9006
9006
9006
so, assuming have following data structure:
$array[index] = psobject following properties: id, data, category, etc.
and want keep id's in same file - grouping id (value) , writing each group of same ids file? work this:
$grouping = $array | group-object id foreach($group in $grouping) { $group.group | export-csv "group $($group.name).csv" }
if want more complex, it'd helpful see data structure , understand how trying use this. if goal have 50k-ish records in each file, might try this:
$grouping = $array | group-object id $tempfilearray = @() $fileid = 1 foreach($group in $grouping) { $tempfilearray += $group.group if ($tempfilearray.count -ge 50000) { $tempfilearray | export-csv "file $($fileid).csv" $fileid++ $tempfilearray = @() } } # export remaining entries if ($tempfilearray.count -gt 0) { $tempfilearray | export-csv "file $($fileid).csv" }
mike
Windows Server > Windows PowerShell
Comments
Post a Comment