8000 cut2 · githubnext/eea@96f8b41 · GitHub
[go: up one dir, main page]

Skip to content

Commit 96f8b41

Browse files
committed
cut2
1 parent a362761 commit 96f8b41

File tree

1 file changed

+8
-1
lines changed

1 file changed

+8
-1
lines changed

docs/report.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,14 @@ In practice, numerous techniques are utilized to "stabilize" the AI code generat
7070

7171
These techniques are all useful, but the problems of instability and ambiguity are endemic to human-iterative AI-generative workflows. These problems must always be navigated in some form by designers of natural-language coding tools, because the user will always want to iterate on the words they use to describe their intent: iterating on words is one of the most essential aspects of human creativity. This is true even when the user is not using natural language, but rather a formal specification language or strict specification methodology.
7272

73-
One approach to this is to switch to a "task-oriented" mode of describing changes to software. However, another more human problem arises with task-oriented natural language programming: the user must "find" the words, the vocabulary, the concepts by which to describe the change they intend. Often the user has no idea how to do this. This problem has been known for many years in information retrieval, referred to as [the "vocabulary problem"](https://dl.acm.org/doi/10.1145/32206.32212). The vocabulary problem arises because the user must find the right words to describe their intent, and this can be difficult, especially when the user is not familiar with the codebase or the domain. Further the words chosen by the user may not be the same as the words used in the codebase. Another recurring problem is the "reference problem" - words must be found to refer to exact code locations and entities in code sufficiently unambiguously. In code this can mean finding natural language to refer to subtle points in the flow of information or control, or to specific data structures or algorithms. In order to change an information flow, it can first be necessary to summarize the whole flow, and then describe exactly how it is to change. This is tedious in natural language.
73+
One approach to this is to switch to a "task-oriented" mode of describing changes to software, and this approach is now becoming normal, for example:
74+
75+
- Copilot Workspace, which allows users to describe tasks in natural language and then generates code to implement those tasks.
76+
- Cursor Compose, which operates similarly in an IDE context.
77+
- VSCode Edits, which is again similar and IDE-based.
78+
- GitHub Spark, which accepts iterative updates to change a generated app.
79+
80+
However, another more human problem arises with task-oriented natural language programming: at each task description the user must "find" the words, the vocabulary, the concepts by which to describe the change they intend. Often the user has no idea how to do this. This problem has been known for many years in information retrieval, referred to as [the "vocabulary problem"](https://dl.acm.org/doi/10.1145/32206.32212). The vocabulary problem arises because the user must find the right words to describe their intent, and this can be difficult, especially when the user is not familiar with the codebase or the domain. Further the words chosen by the user may not be the same as the words used in the codebase. Another recurring problem is the "reference problem" - words must be found to refer to exact code locations and entities in code sufficiently unambiguously. In code this can mean finding natural language to refer to subtle points in the flow of information or control, or to specific data structures or algorithms. In order to change an information flow, it can first be necessary to summarize the whole flow, and then describe exactly how it is to change. This is tedious in natural language.
7481

7582
To summarize the situation: the advent of Generative AI means humans are now finding it compelling and productive to describe software changes with words. However words are ambiguous, the artifacts generated from them are unstable, and human often struggles to find the right words to describe intent sufficiently unambiguously.
7683

0 commit comments

Comments
 (0)
0