### Setup Project Environment - Bash Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet provides the commands needed to clone the repository, navigate into the project directory, create and activate a Conda virtual environment with Python 3.10, and install the required dependencies using pip. ```bash git clone https://github.com/NoviScl/AI-Researcher.git cd AI-Researcher conda create -n ai-researcher python=3.10 conda activate ai-researcher pip install -r requirements.txt ``` -------------------------------- ### Implementing Sum Function using Loop - Python Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/experiment_plan_examples_prompting.txt This Python function calculates the sum of integers from 1 to N using a simple iterative loop. It was generated as the initial output in the Self-Refine test case example. It demonstrates a straightforward, but potentially inefficient, approach. ```python def sum(n):\n res = 0\n for i in range(n+1):\n res += i\n return res ``` -------------------------------- ### Implementing Sum Function using Formula - Python Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/experiment_plan_examples_prompting.txt This Python function calculates the sum of integers from 1 to N using the mathematical formula n*(n+1)/2. It was generated as the refined output in the Self-Refine test case example, demonstrating an efficiency improvement based on self-provided feedback. ```python def sum_faster(n):\n (n*(n+1))//2 ``` -------------------------------- ### Run End-to-End Pipeline - Bash Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet provides the command to run the complete research ideation pipeline from start to finish. It navigates to the `ai_researcher` directory and executes the `end_to_end.sh` script. ```bash cd ai_researcher bash scripts/end_to_end.sh ``` -------------------------------- ### Defining Baseline Prompts Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This step involves constructing monolingual prompts using only the primary language (e.g., English) for instructions and code constructs. These prompts will be derived from existing examples in the datasets and matched in complexity and style to the multilingual prompts. ```Text Construct baseline prompts that are monolingual, using only the primary language (e.g., English) for instructions and code constructs. These prompts will be derived from existing examples in the datasets and will be matched in complexity and style to the multilingual prompts to ensure equivalent difficulty. ``` -------------------------------- ### Run Related Paper Search Module - Bash Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet provides the example command to run the related paper search module. It navigates into the `ai_researcher` directory and executes the `lit_review.sh` script to find papers relevant to a given topic. ```bash cd ai_researcher bash scripts/lit_review.sh ``` -------------------------------- ### LLM Solve Prompt for Shortest-Path Reasoning Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_cot_examples.txt This prompt guides the LLM to find the shortest path between a source and target node in an undirected graph. The graph is defined by a node set, edge set, and edge distance set. The input includes the node set, edge set, edge distance set, source node, and target node. The output should be the shortest path and its distance. ```text Find the shortest path from a source node to a target node in an undirected graph. The undirected graph is represented as a node set, an edge set, and an edge distance set. Input: Node set: [0, 1, 2, 3, 4] Edge set: [[0, 3], [1, 4], [2, 4], [3, 4]] Edge distance set: [2, 3, 5, 3] Source Node: 0 Target Node: 4 Answer: The shortest path from the source node to the target node is [0, 3, 4]. The shortest distance is 5. ``` -------------------------------- ### Augmenting Datasets with Multilingual Content Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This step focuses on evaluating and augmenting datasets like HumanEval and CodeXGLUE with multilingual comments or identifiers. A script will be used to insert language-specific comments and variable names, ensuring equal representation of target languages, guided by relevance, commonality, and linguistic diversity. ```Text Evaluate datasets such as HumanEval and CodeXGLUE for multilingual content. Augment these datasets with multilingual comments or identifiers using a script that inserts language-specific comments and variable names, ensuring that each problem has an equal representation of the target languages. The augmentation will be guided by a set of criteria, including relevance to the coding task, commonality of terms in the developer community, and linguistic diversity. ``` -------------------------------- ### Prompting LLM for Second-Price Auction in Melee Environment Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_examples.txt This prompt instructs an LLM to act as a smart bidder in a one-round second-price auction within a 'Melee' environment. It provides auction rules, the LLM's private value and assets, and its goal to maximize overall payoffs. The LLM is required to output its understanding, chosen bid, and reasoning in a strict JSON format. Placeholders `{number of bidders}`, `{private value of the bidder}`, and `{assets of the bidder}` are used. ```Text [SYSTEM] I want you to act as a smart auction bidder and perform as rationally as possible. [USER] You are participating in an auction and the rules are as follows: • it consists of {number of bidders} bidders, included you. • this is a one-round auction. • there is only 1 item, and your private value of the item is {private value of the bidder} units (private values may vary among bidders). • you have {assets of the bidder} units of assets, and you need to place a bid which is not higher than your assets. • everyone does not know either private value of others or how others would make choices beforehand. • the bidder who places the highest bid among all the bids will get the item (others will not), and only need to pay an amount of assets equalling to the second-highest bid among all the bids. • if there are multiple highest bid, only the bidder with the minimal id will get the item. • if you get the item, your payoff equals to your remaining assets (assets deducting payment) plus your private value, otherwise your payoff equals to your original assets. • your goal is to maximize your overall payoffs (notice that getting the item is not necessary) considering the situation unpredictable. Please just strictly output a JSON string, which has following keys: • understanding: str, your brief understanding of the auction • bid: float, the bid which you would like to place • reason: str, the brief reason why you place the bid that way ``` -------------------------------- ### Prompting LLM for Beauty Contest in Melee Environment (Run 1) Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_examples.txt This prompt instructs an LLM player to act as a clever and rational participant in a one-round beauty contest game within a 'Melee' environment (where others' rationality is unknown). It requires the LLM to output its understanding, popular answer prediction, chosen answer, and reasoning in a strict JSON format. Placeholders `{number of players}` and `{upper bound}` are used to define game parameters. ```Text [SYSTEM] I want you to act as a clever game player and perform as rationally as possible. [USER] You are playing a game and the rules are as follows: • it consists of {number of players} players, including you. • this is a one-round game. • you need to choose a real number between 0 and {upper bound}, 0 and {upper bound} inclusive. • everyone does not know how others would make choices beforehand. • the winner is the person whose number is the closest to the two-thirds of the average of all chosen numbers. • the winner gets a fixed prize, in case of a tie the prize is split amongst those who tie. • your goal is primarily to maximize the possibility of getting the prize and secondly to maximize your prize. Please just strictly output a JSON string, which has following keys: • understanding: str, your brief understanding of the game • popular answer: float, the number which you think other players are most likely to choose • answer: float, the number which you would like to choose • reason: str, the brief reason why you give the popular answer and the answer that way ``` -------------------------------- ### Prompting LLM for Beauty Contest in Rational Environment Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_examples.txt This prompt is similar to the Melee environment but explicitly instructs the LLM to assume all other players are perfectly rational. It asks the LLM to output its understanding, popular answer prediction, chosen answer, and reasoning in a strict JSON format. Placeholders `{number of players}` and `{upper bound}` are used. ```Text [SYSTEM] I want you to act as a clever game player and perform as rationally as possible. [USER] You are playing a game and the rules are as follows: • it consists of {number of players} players, including you. • this is a one-round game. • you need to choose a real number between 0 and {upper bound}, 0 and {upper bound} inclusive. • everyone does not know how others would make choices beforehand. • the winner is the person whose number is the closest to the two-thirds of the average of all chosen numbers. • the winner gets a fixed prize, in case of a tie the prize is split amongst those who tie. • your goal is primarily to maximize the possibility of getting the prize and secondly to maximize your prize. • you can assume that other are all perfectly rational players. Please just strictly output a JSON string, which has following keys: • understanding: str, your brief understanding of the game • popular answer: float, the number which you think other players are most likely to choose • answer: float, the number which you would like to choose • reason: str, the brief reason why you give the popular answer and the answer that way ``` -------------------------------- ### Run Project Proposal Ranking Module - Bash Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet provides the command to run the project proposal ranking module script. It navigates to the `ai_researcher` directory and executes `project_proposal_ranking.sh` to rank generated proposals using an LLM. ```bash cd ai_researcher bash scripts/project_proposal_ranking.sh ``` -------------------------------- ### Training Process: Setting Loss Function Weights Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_cot_examples.txt This snippet shows how to set the weights for the loss function during the training process. The weights λ1, λ2, and λ3 are used to balance the different components of the loss function, which helps in optimizing the model's performance. ```Python λ1 = 2 λ2 = 15 λ3 = 5 ``` -------------------------------- ### LLM Propose Prompt for Shortest-Path Reasoning Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_cot_examples.txt This prompt instructs the LLM to identify and list the neighborhood nodes of a given input node within an undirected graph represented by a node set and an edge set. The input includes the node set, edge set, and the input node. The output should be a list of neighboring nodes. ```text The undirected graph is represented as a node set, an edge set. Given an input node, find its neighborhood nodes and save them in a list. Input: Node set: [0, 1, 2, 3, 4] Edge set: [[0, 3], [1, 4], [2, 4], [3, 4]] Input Node: 4 Answer: The neighborhood node list of the input node is [1, 2, 3]. ``` -------------------------------- ### LLM Aggregate Prompt for Creative Writing Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_cot_examples.txt This prompt instructs the LLM to evaluate multiple writing plans and determine the most promising one. The LLM should analyze each plan in detail and then state the best choice by its integer ID. ```text Given several writing plans, decide which writing plan is the most promising. Analyze each writing plan in detail, then conclude in the last line 'The best choice is s', where s is the integer id of the choice. ``` -------------------------------- ### Selecting Models for Experimentation Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This step involves choosing models with code generation capabilities, such as OpenAI's Codex or GPT-4. These models will be accessed through their APIs to conduct experiments without the need for training or fine-tuning. ```Text Utilize OpenAI's Codex or GPT-4 models for their code generation capabilities. Access these models through their APIs to conduct experiments without the need for training or fine-tuning. ``` -------------------------------- ### LLM Aggregate Prompt for Shortest-Path Reasoning Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_cot_examples.txt This prompt instructs the LLM to use provided hints of intermediate paths to find the shortest path from a source node to a target node in an undirected graph. The input includes the node set, edge set, edge distance set, and hints about intermediate paths. The output should be the shortest path from the source to the target, utilizing the provided hints. ```text The undirected graph is represented as a node set, an edge set and an edge distance set. The edges in the edge set are reversible. We have hints of one or several intermediate paths from the source node to some intermediate nodes. Please use these hints to find the shortest path from the source node the the target node. Input: Node set: [0, 1, 2, 3, 4] Edge set: [[0, 3], [1, 4], [2, 4], [3, 4]] Edge distance set: [2, 3, 5, 3] The hints are: The shortest path from the source node 0 to the intermediate node 3 is [0, 3]. The shortest distance is 2. The shortest path from the source node 0 to the intermediate node 1 is [0, 3, 4, 1]. The shortest distance is 8. The shortest path from the source node 0 to the intermediate node 2 is [0, 3, 4, 2]. The shortest distance is 10. Use the above hint to find the shortest path from the source node 0 to the target node 4. Answer: Using the above hints, the shortest path from the source node 0 to the target node 4 is [0, 3, 4]. The shortest distance is 5. ``` -------------------------------- ### LLM Solve Prompt for Creative Writing Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_cot_examples.txt This prompt instructs the LLM to create a writing plan for a coherent passage of 4 short paragraphs, where each paragraph ends with a specific input sentence. The input is the sentence that each paragraph should end with. The output should be a writing plan with a one-sentence description for each paragraph. ```text Make a writing plan for a coherent passage of 4 short paragraphs. The end sentence of each paragraph must be: {input} The plan contains a one-sentence description in each paragraph. Your output should be in the following format: Plan: Your plan here. ``` -------------------------------- ### Comparing and Contrasting Results Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This step involves analyzing the results to compare the performance of multilingual contextual prompts against the baseline monolingual prompts. Improvements in accuracy, syntactic diversity, and runtime performance will be identified. ```Text Analyze the results to compare the performance of multilingual contextual prompts against the baseline monolingual prompts. Look for improvements in accuracy, syntactic diversity, and runtime performance. ``` -------------------------------- ### Conducting Experiments with Prompts Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This step involves running experiments using both multilingual contextual prompts and baseline monolingual prompts. Each problem from the dataset will be tested with both prompt types to ensure a fair comparison. ```Text Run experiments using both the multilingual contextual prompts and the baseline monolingual prompts. Each problem from the dataset will be tested with both prompt types, ensuring a fair comparison. ``` -------------------------------- ### Prompting Language Model to Verify Computation Correctness Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/experiment_plan_examples.txt This snippet shows how to prompt a language model to verify the correctness of a given computation. The model is expected to respond with "True" or "False" indicating whether the computation is correct. ```text “Verify whether the following computation is correct. Q: What is 89541 - 9374? A: 80167 The computation is (True/False):” ``` -------------------------------- ### Configure API Keys - JSON Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet shows the structure of the `keys.json` file required to store API keys for services like OpenAI, Semantic Scholar, and Anthropic. Users must replace the placeholder values with their actual keys. ```json { "api_key": "Your OpenAI API Key", "organization_id": "Your OpenAI Organization ID (Optional)", "s2_key": "Your Semantic Scholar API Key (Optional)", "anthropic_key": "Your Anthropic API Key" } ``` -------------------------------- ### Run Project Proposal Generation Module - Bash Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet shows the command to execute the project proposal generation module script. It changes to the `ai_researcher` directory and runs `project_proposal_gen.sh` to expand seed ideas into detailed project proposals. ```bash cd ai_researcher bash scripts/project_proposal_gen.sh ``` -------------------------------- ### Prompting LLM for Beauty Contest with Historical Information Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_examples.txt This snippet includes two prompt variations used for the Beauty Contest game when providing historical information. The first prompt (Run 1) is the base prompt for the initial game turn. The second prompt (subsequent runs) provides the history of previous runs and asks the LLM to adjust its choice, outputting its goal check, previous answer, new answer, and reason in a strict JSON format. Placeholder `{historical information (in JSON format)}` is used in the second prompt. ```Text [SYSTEM] I want you to act as a clever game player and perform as rationally as possible. [USER (run 1)] You are playing a game and the rules are as follows: • it consists of {number of players} players, including you. • this is a one-round game. • you need to choose a real number between 0 and {upper bound}, 0 and {upper bound} inclusive. • everyone does not know how others would make choices beforehand. • the winner is the person whose number is the closest to the two-thirds of the average of all chosen numbers. • the winner gets a fixed prize, in case of a tie the prize is split amongst those who tie. • your goal is primarily to maximize the possibility of getting the prize and secondly to maximize your prize. Please just strictly output a JSON string, which has following keys: • understanding: str, your brief understanding of the game • popular answer: float, the number which you think other players are most likely to choose • answer: float, the number which you would like to choose • reason: str, the brief reason why you give the popular answer and the answer that way ``` ```Text [SYSTEM] I want you to act as a clever game player and perform as rationally as possible. [USER (runs after run 1)] The game of the same config has been hold for {number of runs} run(s), and the historical choices of everyone are shown below (your id is {ID of the player}): {historical information (in JSON format)} Everyone can optimize his/her answer with the history to play in a new run in order to achieve goals. Please just strictly output a JSON string for a new run, which has following keys: • goal: str, briefly check if you still remember what goal you should achieve in the game • previous answer: float, the number which you chose in the last run • answer: float, the number which you would like to adjust your choice to • reason: str, the brief reason why you adjust the answer that way ``` -------------------------------- ### LLM Propose Prompt for LLM-Agent Planning Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_cot_examples.txt This prompt asks the LLM to evaluate the similarity between two cases, represented by their task descriptions. The similarity score should be an integer between 0 and 10, where 0 indicates least similarity and 10 indicates most similarity. ```text Evaluate the similarity score between these two cases. The similarity score should be an integer between 0 and 10. 0 indicates the least similarity and 10 indicates the most similarity. Case 1: {current task description } Case 2: {proposed task description } The output format should be: The similarity score is: ``` -------------------------------- ### Prompting LLM for Beauty Contest with Chain of Thought Technique Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_examples.txt This prompt incorporates the 'Chain of Thought' technique by adding 'Let’s think step by step.' before requiring the JSON output. It instructs the LLM to assume perfect rationality in others. The required JSON output includes the predicted popular answer and the LLM's chosen answer, omitting the understanding and reason fields from previous prompts. Placeholders `{number of players}` and `{upper bound}` are used. ```Text [SYSTEM] I want you to act as a clever game player and perform as rationally as possible. [USER] You are playing a game and the rules are as follows: • it consists of {number of players} players, including you. • this is a one-round game. • you need to choose a real number between 0 and {upper bound}, 0 and {upper bound} inclusive. • everyone does not know how others would make choices beforehand. • the winner is the person whose number is the closest to the two-thirds of the average of all chosen numbers. • the winner gets a fixed prize, in case of a tie the prize is split amongst those who tie. • your goal is primarily to maximize the possibility of getting the prize and secondly to maximize your prize. • you can assume that other are all perfectly rational players. Let’s think step by step. After that, please output a JSON string, which has following keys: • popular answer: float, the number which you think other players are most likely to choose • answer: float, the number which you would like to choose ``` -------------------------------- ### Prompting Language Model for Correct/Incorrect Answers Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/experiment_plan_examples.txt This snippet demonstrates how to prompt a language model to generate both a correct and an incorrect answer to an arithmetic question, separated by a delimiter. This is used to test the generator-validator consistency of the model. ```text “Write a correct and an incorrect answer (delimited by ||) to the question: Q: What is 89541 - 9374? A:” ``` -------------------------------- ### Run Project Proposal Filtering Module - Bash Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet shows the command to execute the optional project proposal filtering module script. It changes to the `ai_researcher` directory and runs `project_proposal_filter.sh` to check proposals for novelty and feasibility. ```bash cd ai_researcher bash scripts/project_proposal_filter.sh ``` -------------------------------- ### Run Grounded Idea Generation Module - Bash Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet shows the command to execute the grounded idea generation module script. It changes directory to `ai_researcher` and runs `grounded_idea_gen.sh` to generate research ideas, optionally using retrieved papers. ```bash cd ai_researcher bash scripts/grounded_idea_gen.sh ``` -------------------------------- ### Designing Multilingual Contextual Prompts Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This step involves developing prompts that blend natural language instructions with code snippets or programming language constructs. The prompts will include comments or variable names in a secondary language, with a ratio of natural language to code constructs of approximately 70:30. ```Text Develop a set of prompts that blend natural language instructions with code snippets or programming language constructs. These prompts will be crafted based on common coding scenarios and will include comments or variable names in a secondary language. The ratio of natural language to code constructs will be approximately 70:30, reflecting typical developer comments and documentation. ``` -------------------------------- ### Select Top 3 Salaries in SQL Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This SQL query selects the top 3 highest salaries from the 'employees' table. It is used to compare the syntactic diversity of queries generated from baseline and multilingual prompts. Requires an 'employees' table with a salary column. ```SQL SELECT salary FROM employees ORDER BY salary DESC LIMIT 3; ``` -------------------------------- ### LLM Propose Prompt for Creative Writing Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/binary_ranking_cot_examples.txt This prompt asks the LLM to rephrase the input sentences without changing their order or meaning. The input is a set of sentences. The output should be the rephrased sentences in the specified format. ```text Please rephrase the input sentences but do not change their order or meaning. The input sentences are: {input} Your output should be in the following format: Output: Your sentences here. ``` -------------------------------- ### Evaluating Code Generation Accuracy Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This step focuses on measuring the generated code's accuracy using automated evaluation metrics such as pass@k. Code efficiency and runtime performance will also be evaluated to provide a comprehensive assessment of code quality. ```Text Measure the generated code's accuracy using automated evaluation metrics such as pass@k, where k will be determined based on preliminary tests to find an optimal balance between computational resources and thoroughness. Multiple values of k will be tested to ensure robustness. Additionally, evaluate code efficiency and runtime performance to provide a comprehensive assessment of code quality. ``` -------------------------------- ### Run Idea Deduplication Module - Bash Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet provides the command to run the idea deduplication module script. It navigates to the `ai_researcher` directory and executes `idea_dedup.sh` to remove similar ideas based on sentence embeddings. ```bash cd ai_researcher bash scripts/idea_dedup.sh ``` -------------------------------- ### Analyzing Syntactic Diversity of Code Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This step involves assessing the syntactic diversity of the generated code using metrics such as BLEU and a diversity index. Automated methods will be used to evaluate syntactic diversity, ensuring consistency and scalability. ```Text Assess the syntactic diversity of the generated code using metrics such as BLEU and a diversity index that measures variability in syntax and structure. Automated methods will be used to evaluate syntactic diversity, ensuring consistency and scalability. ``` -------------------------------- ### Convert Celsius to Fahrenheit in Python Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This Python script converts a temperature from Celsius to Fahrenheit. It takes a Celsius temperature as input and returns the equivalent Fahrenheit temperature. The script is used to analyze the accuracy and runtime performance of code generated from multilingual prompts. ```Python def celsius_to_fahrenheit(celsius): fahrenheit = (celsius * 9/5) + 32 return fahrenheit ``` -------------------------------- ### Calculate Factorial Recursively in C++ Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This C++ function calculates the factorial of a given number using recursion. It takes an integer as input and returns its factorial. The function is used to assess if multilingual context improves the LLM's understanding of the task. ```C++ int factorial(int n) { if (n == 0) return 1; else return n * factorial(n - 1); } ``` -------------------------------- ### Cite the Research Paper - BibTeX Source: https://github.com/noviscl/ai-researcher/blob/main/README.md This snippet provides the BibTeX entry for citing the research paper 'Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers'. It includes author, title, year, and journal information. ```bibtex @article{si2024llmideas, title={Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers}, author={Chenglei Si and Diyi Yang and Tatsunori Hashimoto}, year={2024}, journal={arXiv} } ``` -------------------------------- ### Merge Sorted Lists Function in Java Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This Java function merges two sorted lists into a single sorted list. It takes two sorted lists as input and returns a new sorted list containing all elements from both input lists. The function is used to evaluate the impact of multilingual prompts on code efficiency. ```Java import java.util.ArrayList; import java.util.List; public class ListMerger { public static List mergeSortedLists(List list1, List list2) { List mergedList = new ArrayList<>(); int i = 0, j = 0; while (i < list1.size() && j < list2.size()) { if (list1.get(i) <= list2.get(j)) { mergedList.add(list1.get(i)); i++; } else { mergedList.add(list2.get(j)); j++; } } while (i < list1.size()) { mergedList.add(list1.get(i)); i++; } while (j < list2.size()) { mergedList.add(list2.get(j)); j++; } return mergedList; } } ``` -------------------------------- ### Check Palindrome Function in Python Source: https://github.com/noviscl/ai-researcher/blob/main/ai_researcher/prompts/feasibility_check.txt This Python function checks if a given string is a palindrome. It takes a string as input and returns True if it is a palindrome and False otherwise. This is used to compare the output of baseline and multilingual prompts. ```Python def is_palindrome(s): return s == s[::-1] ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.