5 resources

  • Liu
    |
    Aug 4th, 2024
    |
    conferencePaper
    Liu
    Aug 4th, 2024
  • Yiqing Xie, Alex Xie, Divyanshu Sheth
    |
    Mar 31st, 2024
    |
    preprint
    Yiqing Xie, Alex Xie, Divyanshu Sheth
    Mar 31st, 2024

    To facilitate evaluation of code generation systems across diverse scenarios, we present CodeBenchGen, a framework to create scalable execution-based benchmarks that only requires light guidance from humans. Specifically, we leverage a large language model (LLM) to convert an arbitrary piece of code into an evaluation example, including test cases for execution-based evaluation. We illustrate the usefulness of our framework by creating a dataset, Exec-CSN, which includes 1,931 examples...

  • Yiqing Xie, Alex Xie, Divyanshu Sheth
    |
    Mar 31st, 2024
    |
    preprint
    Yiqing Xie, Alex Xie, Divyanshu Sheth
    Mar 31st, 2024

    To facilitate evaluation of code generation systems across diverse scenarios, we present CodeBenchGen, a framework to create scalable execution-based benchmarks that only requires light guidance from humans. Specifically, we leverage a large language model (LLM) to convert an arbitrary piece of code into an evaluation example, including test cases for execution-based evaluation. We illustrate the usefulness of our framework by creating a dataset, Exec-CSN, which includes 1,931 examples...

  • Okan Bulut, Maggie Beiting-Parrish, Jodi...
    |
    Jun 27th, 2024
    |
    preprint
    Okan Bulut, Maggie Beiting-Parrish, Jodi...
    Jun 27th, 2024

    The integration of artificial intelligence (AI) in educational measurement has revolutionized assessment methods, enabling automated scoring, rapid content analysis, and personalized feedback through machine learning and natural language processing. These advancements provide timely, consistent feedback and valuable insights into student performance, thereby enhancing the assessment experience. However, the deployment of AI in education also raises significant ethical concerns regarding...

  • Abhimanyu Dubey, Abhinav Jauhri, Abhinav...
    |
    Aug 15th, 2024
    |
    preprint
    Abhimanyu Dubey, Abhinav Jauhri, Abhinav...
    Aug 15th, 2024

    Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language...

Last update from database: 04/04/2025, 19:15 (UTC)
Powered by Zotero and Kerko.