ghh
a year ago
Wow, their example to "clean up the code" does a bit more than just refactoring to make code more readable, it appears to change the output.
One would have to check the resulting code carefully to see if the meaning is still as originally intended, or replaced with code that is more probable to be correct (but no longer working).
For instance, it replaces this:
if dataset == 'animals':
if dataset == 'turtle':
x_train, y_train, x_test, y_test = datasets.load_turtles(with_bowtie=False)
elif dataset = 'formal_turtle':
x_train, y_train, x_test, y_test = datasets.load_turtles(with_bowtie=True)
else:
if dataset == 'turtle':
x_train, y_train, x_test, y_test = datasets.load_turtles(with_bowtie=False)
elif dataset == 'formal_turtle':
x_train, y_train, x_test, y_test = datasets.load_turtles(with_bowtie=True)
Hasnep
a year ago
Actually, the code before does nothing if dataset is set to 'animals', 'turtle' or 'formal_turtle', most of the branches are inaccessible. Also, the extra else clause that raises an error and the line
elif dataset = 'formal_turtle':
I think 'clean up' here means something closer to 'convert this to what I'm trying to write'.
_endif_
a year ago
Agreed, but I have to say data cleaning is actually one of the hardest step, LLMs are simply not there yet.
It's almost impossible to for LLM to tell all the invalid rows at once since the data cannot be fit into the context window. If we prompt the model to thoroughly do data cleaning, there will be many try-and-fail steps. This happens to me as a human, I clean some rows, expect my program to run with the data, only to find there are more malformed data. LLM cannot get it right for now, actually I see many cases that LLM fails because it wants to convert types (e.g. string to date).
Based on my experience, the best way is simply to skip the data cleaning step in the planning stage (you can provide feedback asking the tool to not do any steps).
IamLoading
a year ago
Unfortunately, this is true with most LLMs.