Scalytics Connect Release Notes

Scalytics Connect

Apache Wayang ist das Herzstück unserer Produkte. Apache Wayang (Incubating) ist die einzige plattformübergreifende Open-Source-Datenverarbeitungs-Engine. Anwendungsentwickler spezifizieren Anwendungen mithilfe der API von Apache Wayang.
‍

What's New (March 2025)

• Enhanced GPU Management & Resource Optimization:
  • Advanced GPU assignment optimizer and worker system, significantly improving model inference performance and resource utilization.
  • Seamless GPU assignment management.
  • Intelligent model reload functionality for GPU assignment changes, ensuring optimal resource allocation.
  • GPU assignment backend and interference router for improved stability and performance.
  • Safeguards against multiple GPU assignments for a single model, preventing resource conflicts.
  • Manual admin controls for active model cache, providing greater flexibility for enterprise deployments.
• Advanced Model Streaming & Chat Capabilities:
  • Real-time streaming responses from local models, dramatically improving user experience and interaction fluidity.
  • Chat routing and streaming system, enhancing reliability of AI interactions.
  • Persistence mechanisms for chat sessions, ensuring data continuity and improved recovery.
• Intelligent Model Management:
  • Sophisticated model-specific response filtering in the prompting system, delivering more relevant AI outputs.
  • Segmented model downloads from Hugging Face, improving deployment reliability.
  • Audit of model download processes, enhancing security and reliability.
• Infrastructure & System Reliability Improvements:
  • Significantly improving system availability.
  • Application state management in callbacks, leading to more robust error handling and recovery.
  • Refined hardware overview display, providing clearer insights into system resources and utilization.

‍

Scalytics Federated AI MCP features:

• Der KI-basierte Optimierer wählt automatisch eine optimale Konfiguration von Klassendatenverarbeitungs-Frameworks wie Java Streams oder Apache Spark aus, auf denen Anwendungen ausgeführt werden.
• Blossom Core führt die Programmausführung durch. Es abstrahiert die verschiedenen plattformspezifischen APIs und koordiniert die plattformübergreifende Kommunikation.
• Anwendungen können auf mehreren Datenverarbeitungsplattformen ausgeführt werden, ohne den systemeigenen Code der zugrunde liegenden Plattformen zu ändern.
• Federated data processing: In-situ processing in different sites without moving raw data outside their origin.
• Build and execute cross platform machine learning pipelines in a unified way.
• NEW: Federated Machine Learning
  • Federated analytics by integrating multiple platforms across silos
  • Developers: Train ML models using federated learning in a platform agnostic way
• NEW: Supporting unsupervised learning (e.g., using K-means) and Stochastic Gradient Decent optimization technique for Federated Learning across supported data platforms
• NEW: Auditing compliance (who accessed what when) and training audits (basic)

‍

Data sources:

• PostgresSQL
• Columnar Data Files (e.g., CSV, Iceberg, Parquet, ORC)
• SQlite (e.g. Mobiles, Embedded)
• Local file systems
• Distributed file systems (e.g., HDFS, S3)
• Apache Kafka
• NEW: Remote files over http(s)
• NEW: JDBC based data sources

‍

Data Processing Platforms:

• Java 8 Streams
• Apache Spark / DataBricks
• Postgres
• SQLite
• Apache Flink / Confluent, Decodable
• NEW: Apache Kafka
• NEW: Tensorflow
• NEW: JDBC based platforms

‍

Programming APIs

• Java
• Scala
• Basic SQL
• New: Python (limited support)

‍

Runtime

• NEW: Actor-based runtime for building federated applications

‍

This release represents our ongoing commitment to delivering a secure, efficient, and privacy-focused AI platform for enterprise environments. We continue to enhance Scalytics Connect based on customer feedback and emerging industry requirements, ensuring our solution remains at the forefront of enterprise AI technology.

‍