Discussions

• 1 You must be logged in to vote

  🙏

  If I quantize a model with HQQ and pack it in Marlin format, will that work?

  BlairSadewitz asked Dec 13, 2024 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  💡

  support for arm64

  BlairSadewitz started Mar 11, 2024 in Ideas

  

  

  

  

  27
• 1 You must be logged in to vote

  💡

  Include "exclude top choices" sampler

  BlairSadewitz started Aug 18, 2024 in Ideas · Closed

  

  

  1
• 3 You must be logged in to vote

  🙌

  Aphrodite Benchmark Tests in two 2x3090 rigs

  miku448 started Dec 5, 2023 in Show and tell

  

  

  

  

  5
• 1 You must be logged in to vote

  🙏

  Does aphrodite-engine support models quantized with vllm's llm-compressor?

  BlairSadewitz asked Jul 31, 2024 in Q&A · Answered

  

  

  3
• 1 You must be logged in to vote

  🙏

  How to unload and change models for local offline inferencing with Aphrodite?

  murtaza-nasir asked Jun 13, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Good front end and any way to quickly load models without using terminal?

  Kaszebe asked Aug 12, 2024 in Q&A · Unanswered

  

  

  

  

  6
• 1 You must be logged in to vote

  💬

  Possibilities wrt. intel GPUs (et. al.); SYCL / OpenCL / Vulkan etc.?

  ghchris2021 started Jul 3, 2024 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  Ideal value for GPU_MEMORY_UTILIZATION

  houmie asked Jun 12, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Selecting a specific GPU

  gsuhm asked Mar 6, 2024 in Q&A · Unanswered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  GPU Configuration, Concurrent Request Handling & RoPE scaling

  houmie asked Jun 1, 2024 in Q&A · Unanswered

  

  

  

  

  6
• 2 You must be logged in to vote

  🙏

  How to enable 4-Bit Cache (Q4 Cache)?

  houmie asked May 8, 2024 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  What kind of speed should I expect from a P40 and many P40 ？

  sorasoras asked Apr 30, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  How to ensure that it consumes all "Pending" requests so that it doesn't launch a process without using all of "--max-num-seqs"?

  Lucidology started Apr 17, 2024 in General

  

  

  3
• 1 You must be logged in to vote

  🙏

  update-runtime.sh freezes computer

  Lucidology asked Apr 19, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Is there a way to download models without engaging GPU memory?

  murtaza-nasir asked Apr 17, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Throughput not scaling with additional GPUs

  murtaza-nasir asked Apr 7, 2024 in Q&A · Answered

  

  

  

  

  10
• 1 You must be logged in to vote

  💡

  options to disable ipv6 and bind

  BlairSadewitz started Feb 20, 2024 in Ideas

  

  

  0
• 2 You must be logged in to vote

  💬

  AMD Support

  userbox020 started Feb 8, 2024 in General

  

  

  3
• 1 You must be logged in to vote

  💬

  Chat Templates

  AWAS666 started Jan 1, 2024 in General

  

  

  1
• 1 You must be logged in to vote

  💡

  support for [optionally] using hf_transfer to download model

  BlairSadewitz started Dec 8, 2023 in Ideas

  

  

  7
• 1 You must be logged in to vote

  🙏

  CUDA 12 or 11.8 for dev?

  BlairSadewitz asked Dec 7, 2023 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  📣

  Licensing issues with Aphrodite Engine

  discussion

  AlpinDale announced Oct 4, 2023 in Announcements · Closed

  

  

  

  

  15