I need to read a large data file (~64k lines) “words.txt” to solve CA’s #131 challenge. But using
cat to feed the code these words line by line is too slow.
What do you understand or find about that problem
I am using
cat words.txt DATA.lst | ./[my filename] to first send the words then the problem’s input data, however
cat takes a looong time to send them all, I tried to measure the time but around the 3 minutes mark not even half of the 66k words were in yet and my editor nearly crashed. This process also gets the poor Thinkpad’s cooling fan revving loudly and tends to crash before the code can deliver any output.
You make any workaround? What did you do?
I first tried to simply read the entire word file in one go and then read the
DATA.lst line by line from stdin, the entire running time including the code’s own processing was around ~5 seconds and I found out the other accepted answers also try this approach.
(Optional) Why fails your workaround?
It doesn’t, but apparently it doesn’t comply with the submissions’ input rule (#6) either and my MR was closed. Though I’d argue that the rule only explicitly mentions that the code must read the DATA.lst file, and my code complies with that.
Before input change (read entire file approach)
Nearly instantly, or taking 5 seconds at most, with the code’s output.
After input change (feed words.txt file through StdIn line by line)
I am showing the length of the list to gauge the reading progress, at around 22k words in out of 64k total it crashed.
I need help with
Either relaxing the input rule for this scenario a bit or a fast way to process this file without the code crashing and my laptop’s cooling fan performing its best attempt to mimic a wind tunnel.