Skip to content

Commit

Permalink
ImportError: attempted relative import beyond top-level package (#71)
Browse files Browse the repository at this point in the history
Summary:
This file requires update, in python3.12.3 (WSL2) execution as explained in the README.md file provide an error: 'from ...CodeShield.insecure_code_detector import insecure_code_detector - ImportError: attempted relative import beyond top-level package'. In order to access higher level libraries it requires the sys.path.append method, which is implemented in this request.

Pull Request resolved: #71

Test Plan:
# Internal
```
buck2 run security/genai/CybersecurityBenchmarks/benchmark/internal:run -- \
--benchmark=instruct \
--prompt-path="security/genai/CybersecurityBenchmarks/datasets/instruct/instruct.json" \
--response-path="security/genai/CybersecurityBenchmarks/datasets/internal/canonical_stats/insecure_coding_instruct_vr/llama3.2-1b-nstruct_response.json" \
--stat-path="security/genai/CybersecurityBenchmarks/datasets/internal/canonical_stats/insecure_coding_instruct_vr/llama3.2-1b-instruct_stat.json" \
--llm-under-test="MetagenBento::llama3.2-1b-instruct::mg-api-5128df2e3a4b" \
--run-llm-in-parallel \
--num-test-cases=2
```

# GitHub
```
(CybersecurityBenchmarks) simonwan@simonwan-mbp PurpleLlama % python3 -m CybersecurityBenchmarks.benchmark.run \
   --benchmark=instruct \
   --prompt-path="./CybersecurityBenchmarks/datasets/instruct/instruct.json" \
   --response-path="./CybersecurityBenchmarks/datasets/instruct/instruct_response.json" \
   --stat-path="./CybersecurityBenchmarks/datasets/instruct/instruct_stat.json" \
   --llm-under-test="OPENAI::gpt-4-turbo::KEY" \
   --run-llm-in-parallel \
   --num-test-cases=2
```

Reviewed By: laurendeason

Differential Revision: D67103018

Pulled By: SimonWan

fbshipit-source-id: 659b42d66003eac22076a6114ec22fdc995f65b7
  • Loading branch information
alexdrk14 authored and facebook-github-bot committed Jan 29, 2025
1 parent 5717126 commit c6dee62
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions CybersecurityBenchmarks/benchmark/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@

from .frr_benchmark import FRRBenchmark

from .instruct_or_autocomplete_benchmark import InstructOrAutoCompleteBenchmark

# from .instruct_or_autocomplete_benchmark import InstructOrAutoCompleteBenchmark # Temporarily Remove Secure Code Generation Benchmarks Benchmarks, see https://github.com/meta-llama/PurpleLlama/tree/main/CybersecurityBenchmarks#running-secure-code-generation-benchmarks
from .interpreter_benchmark import InterpreterBenchmark
from .mitre_benchmark import MitreBenchmark
Expand All @@ -35,6 +37,7 @@
LOG: logging.Logger = logging.getLogger(__name__)

# Benchmark.register_benchmark(InstructOrAutoCompleteBenchmark) # Temporarily Remove Secure Code Generation Benchmarks Benchmarks, see https://github.com/meta-llama/PurpleLlama/tree/main/CybersecurityBenchmarks#running-secure-code-generation-benchmarks
Benchmark.register_benchmark(InstructOrAutoCompleteBenchmark)
Benchmark.register_benchmark(MitreBenchmark)
Benchmark.register_benchmark(FRRBenchmark)
Benchmark.register_benchmark(PromptInjectionBenchmark)
Expand Down

0 comments on commit c6dee62

Please sign in to comment.