Prompt engineering has recently sprung up. It is how you ask a Copilot to do something. It is like summarizing and explaining what to do. Microsoft have created a set of resources to guide you through the art and science of it.
The full guide on the art and science of prompting. There are useful featured prompts for generating ideas, help me write an email etc.
The Prompting do's and don't can be found here.
Metaprompt design is crucial for generative AI applications.
The process of grounding or retrieval augmented generation (RAG) is the process of grounding your model on relevant context. This grounding can significantly improve the overall accuracy and relevance of the model outputs.
Azure AI Studio can help you quickly and securely ground models on structured, unstructured, and real-time data, including data within Microsoft Fabric.
'A metaprompt, or system message, is a set of natural language instructions used to guide an AI system’s behavior (do this, not that)'. (see below)
The grounding data should be used by the metaprompt and enforce rules that help stop harmful content generation or user manipulations like jailbreaks or prompt injections.
Summary of responsible AI best practices for a metaprompt
ProMetaprompt engineering techniques
Infuse responsible AI tools and practices in your LLMOps
Watch the 45-minute Introduction to Azure AI Studio course on Microsoft Learn.
I came across this excellent resource, the Microsoft Copilot Studio implementation guide. It covers
a 360-degree review of your project covering
Reference architecture examples
Download it from https://lnkd.in/d5h-binr
The second meeting of the Bath Azure User Group was on Thu, Dec 7, 2023, 6:00 PM - 8:30 PM held at Cloud Direct, The Vaults, 1 Bartlett Street, Bath, England, GB, BA1 2QZ
The December meeting had an exciting packed agenda:
Unpacking the Ignite Book of News - Richard Cheney, Luke Fussell
Boosting your Azure CLI with OpenAI - Ira Rainey
The Most Valuable Professional Program - Dr Victoria Holt
I talked about the the program helping the community learn and grow and the passion for working with cutting-edge technology.
There lots of Pizza and plentiful Azure discussions. A fun night.
We are pleased to announce that Data Toboggan will return for its 4th year on Saturday 3rd February 2024. Our 2 track virtual conference covering covering all aspects of Azure Synapse Analytics and Fabric including Dedicated SQL Pools, Serverless SQL Pools, Spark, Pipelines, Data Engineering, Power BI, Machine Learning, and much more!
Imaged created in https://designer.microsoft.com/
.png)
Live view in Microsoft Purview is in preview. Live view is a place that users with data access permissions can find Azure resources and their data assets in the Microsoft Purview Data Catalog without set-up or scanning. The metadata of the resources is immediately available when you access the free or enterprise versions of Microsoft Purview.
There are some limitations for live view for Microsoft Fabric:
Microsoft Fabric items that will be available in the Microsoft Purview Data Catalog using live view are:
Lesson Link Course Introduction - How to Take This Course | Concepts Taught Tech setup and course structure | Learning Goal Setting you up for success while learning in this course | |
| Introduction to Generative AI and LLMs | Concept: Generative AI and the current technology landscape | Understanding what Generative AI is and how Large Language Models (LLMs) work. | |
| Exploring and comparing different LLMs | Concept: Testing, iterating, and comparing different Large Language Models | Select the right model for your use case | |
| Using Generative AI Responsibly | Concept: Understanding the limitations of foundation models and the risks behind AI | Learn how to build Generative AI Applications responsibly | |
| Understanding Prompt Engineering Fundamentals | Code/Concept: Hands-on application of Prompt Engineering Best Practices | Understand prompt structure & usage | |
| Creating Advanced Prompts | Code/Concept: Extend your knowledge of prompt engineering by applying different techniques to your prompts | Apply prompt engineering techniques that improve the outcome of your prompts. | |
| Building Text Generation Applications | Code: Build a text generation app using Azure OpenAI | Understand how to efficiently use tokens and temperature to vary the model's output | |
| Building Chat Applications | Code: Techniques for efficiently building and integrating chat applications. | Identify key metrics and considerations to effectively monitor and maintain the quality of AI-powered chat applications | |
| Building Search Apps Vector Databases | Code: Semantic vs Keyword search. Learn about text embeddings and how they apply to search | Create an application that uses Embeddings to search for data. | |
| Building Image Generation Applications | Code: Image generation and why it's useful in building applications | Build an image generation application | |
| Building Low Code AI Applications | Low Code: Introduction to Generative AI in Power Platform | Build a Student Assignment Tracker App for our education startup with Low Code | |
| Integrating External Applications with Function Calling | Code: What is function calling and its use cases for applications | Setup a function call to retrieve data from an external API | |
| Designing UX for AI Applications | Concept: Designing AI Applications for Trust and Transparency | Apply UX design principles when developing Generative AI Applications | |
| Continue Your Learning | Links to continue your learning from each lesson! | Mastering your Generative AI skills |
I watched the session
Make your data AI ready with Microsoft Fabric and Azure Databricks
Jessica Hawk, Amir Netz, Arun Ulagaratchagan, Patrick Baumgartner, Profile picture of Kristen Christensen, Justyna Lucznik
'Bring your data into the era of AI with Microsoft Fabric, a powerful all in one AI powered analytics solution for enterprises that covers everything from data movement to data science, real time analytics and business intelligence. Learn how Azure Databricks and Microsoft Fabric seamlessly work together to offer customers a modern, price performant analytics solution that helps teams turn data into a competitive advantage.'
The discussion centred around the general availability of Microsoft Fabric. Below are are few slide images from the presentation.
Here is some more information about the changes.
Prepare your data for AI innovation with Microsoft Fabric—now generally available
Microsoft Fabric, explained for existing Synapse users
The new Microsoft Purview portal helps you discover and access data security, data governance, and risk and compliance solutions for all your data in your data estate. This unified experience streamlines navigation for all Purview solutions and provides a single-entry point for settings, search, and roles and permissions management. The risk and compliance solutions in the portal currently opens these solutions in the classic Microsoft Purview compliance portal . The portal inclusions:
New Microsoft Purview portal (preview) can be found at: https://purview.microsoft.com
The solution cards on the portal home page provide quick access to open the Microsoft Purview solutions where access has been granted. It is great to see the Purview portals coming together under one roof as this will make navigation easier.
An exciting introduction to Microsoft Ignite November 15 - 17, 2023. Two themes that I am excited about are the AI and Microsoft Fabric capability.
Microsoft Ignite 2023: AI transformation and the technology driving change
The full list of changes at Microsoft Ignite are listed in the Microsoft Ignite Book of News.
In the main keynote Satya announced over 100 changes. A few of the announcements discussed in the keynote are mentioned here:
Azure Maia - The Microsoft Cloud accelerator optimized for LLM training and inference
Azure Open AI Service updates
Model catalog - New
Models as a Service - announcing
Phi-2 - Announcing
The Phi-2 model is changing the game in artificial intelligence (AI) and it is designed to make machines think more like humans and do it safely.
Azure AI Search - Deliver accurate, hyper-personalized responses in your Gen AI applications
There was also announcement of Mesh on Microsoft Teams being generally available.
Then finally something different Azure Quantum Elements - purpose built to accelerate scientific discovery.
The PASS Data Community Summit 2023 global data conference is happening in Seattle between 14-17 November 14-17 2023.
The first keynote 'Limitless growth, limitless opportunities: Data and innovation in an AI world' was given by Shireesh Thota. He started talking about new technologies and how they offer limitless potential with AI, Generative AI and Real-time capabilities. The limitiless growth with Azure and those SQL databases: SQL Server, Azure SQL, Postgres, CosmosDB and MySQL. These offer limitless scale, flexible tools, Built-in AI, Performance and Guaranteed availability.
There are celebrations a foot due to SQL Server on Windows reaching 30 years old this year. SQL Server 2022 is the fastest adopted version of SQL Server with growth on both Windows and Linux. He also mentioned SQL Server enabled by Azure Arc with that cloud manageability for managing your entire SQL Server estate. Announcements were made
Discussions moved to Azure SQL and the family of SQL cloud in edge databases. A further announcement was made about the Azure SQL Managed Instance feature wave , that is now generally available. Azure SQL Managed instance has a free offer in preview for 12 months of free managed instance, available soon.
What's new in Azure SQL Managed Instance?
They talked about that Cloud Modernization Journey: Evaluate, Optimize, Migrate and Modernize with ease.
Another announcement Azure SQL Database Hyperscale will be same price as commercial OSS databases – General Available mid December.
Some further to announcements were shared
Then concluding with the database of the future will be autonomous and intelligent, developer friendly, cost efficient, and mission critical by default.
I had the privilege to present at the The Form Rooms on Tuesday 14 November 2023 at the Microsoft Cloud Innovation and Security Summit run by Cloud Direct.
The Forms Rooms is on a quiet street in London’s chic Covent Garden that was once a historic Victorian school that has been lovingly restored.
My session was on 'The disruptive power of data and AI'. About how Generative AI, Microsoft Fabric and Data Governance are the talk of the town, but without the building blocks of good quality, secure, accessible data, innovation in business is stunted.
The talk covered how Data & AI are providing a catalyst for innovation addressing
1970s where Codd invented the relational database
To the 1990s when ‘Big Data’ became a term to address the ever-increasing large volume of data, with discussions of volume, velocity , variety and veracity. Here we saw the mention of that ever-important reference to the quality of data.
With that grew the ‘Internet of Things’ processing more data from sensors, devices and communication networks.
Behind the scenes world leading database researchers meet every few years to discuss the impact on practice, challenges and emerging applications. The latest output was documented in the Seattle report in 2019. One of the areas discussed was the fact that it was impractical to continue to move volumes of data between systems. We see this fundamental change in Microsoft Fabric.
And today, it is all about ‘Generative AI’ and what it can create...
There were many really interesting sessions during the conference with Innovation, AI and Security intertwined.
Announcing the Data Toboggan - Alpine Coaster Agenda for APAC, EMEA and AMER
The Asilomar AI Principles are a set of 23 guidelines for responsible AI development that aim to ensure safety, security, and rights of individuals and society. They cover various aspects of AI such as research ethics, transparency, and accountability, and are crucial for building trust in AI technology. Here is a summary of the main points:
For more information, you can visit the Future of Life Institute website or read the WIRED article about the Asilomar AI Principles. Some more reading here . Chat used to create the summary.
Microsoft Ignite is next week. There are lots of exciting sessions.
Online: November 15–16, 2023 PT | In Seattle: November 14–17, 2023 PT
The headline Experience AI transformation in action, online from anywhere. The session catalog is packed full of exciting content and the agenda states whether the session will be recorded or not , so helpful when planning your schedule. A few key sessions to watch.
Keynote: Microsoft Ignite opening
Wednesday, November 15 5:00 PM - 5:45 PM Greenwich Mean Time
Satya Nadella will share how Microsoft is creating new opportunity across our platforms in this new era of AI.
Keynote: Becoming an AI-Powered Organization with Microsoft Copilot
Wednesday, November 15 5:45 PM - 6:35 PM Greenwich Mean Time
Join Rajesh Jha, along with Jared Spataro and other leaders to see how Microsoft Copilot will unlock productivity and transform business process for everyone across functions and industries.
Keynote: AI transformation for your organization with the Microsoft Cloud
Thursday, November 16 5:00 PM - 6:15 PM Greenwich Mean Time
Scott Guthrie and other leaders will share how the Microsoft Cloud is uniquely positioned to help our customers transform by building AI solutions and unlocking data insights using the same platform and services that power all of Microsoft’s comprehensive solutions.
Keynote: The Future of Security with AI
Thursday, November 16 6:15 PM - 7:00 PM Greenwich Mean Time
While the new era of AI presents unprecedented opportunities to elevate human potential, it also ushers in a new set of unknowns and risks. In this session, Charlie Bell and Vasu Jakkal will share how Microsoft is delivering AI for security with Security Copilot, and how we are enabling organizations to secure and govern AI with new capabilities.
Keynote: Inside Microsoft AI innovations with Mark Russinovich
Friday, November 17 8:30 PM - 9:30 PM Greenwich Mean Time
Join Mark Russinovich, Azure CTO and Technical Fellow, for an in-depth exploration of Microsoft's AI architecture. Discover the technology behind our massive supercomputers used for foundational model training, efficient infrastructure for serving models, sustainable datacenter design, AI-aware resource management and scheduling, GPU usage optimization techniques, and advancements in confidential computing to safeguard data during processing.
Make your data AI ready with Microsoft Fabric and Azure Databricks
Wednesday, November 15 7:45 PM - 8:30 PM Greenwich Mean Time
Bring your data into the era of AI with Microsoft Fabric, a powerful all in one AI powered analytics solution for enterprises that covers everything from data movement to data science, real time analytics and business intelligence. Learn how Azure Databricks and Microsoft Fabric seamlessly work together to offer customers a modern, price performant analytics solution that helps teams turn data into a competitive advantage.
Upgrading your data estate to Microsoft Fabric and getting started Q&A
Thursday, November 16
9:30 PM - 10:15 PM Greenwich Mean Time
Looking to get started with Microsoft Fabric, but don't know where to start? Join our product experts to discuss everything from basic set up steps to migrating to Fabric from Synapse to starting a POC with departments where you want to gain traction quickly.
All timings are in Pacific time, and the sessions are:
Wednesday November 15
8.15-9.30am – KEYNOTE: Limitless growth, limitless opportunities: Data and innovation in an AI world
9.30-10.15am – break
10.15-11.30am – COMMUNITY: What’s New in Azure SQL and SQL Server, Data Exposed Live
11.30am-12pm – break
12-1pm – LUNCHEON: Database DevOps Luncheon: All you Ever Wanted to Know but were too Afraid to Ask
1-1.30pm – break
1.30-2.45pm – MICROSOFT: Data Warehousing Simplified in Microsoft Fabric
2.45-3.15pm – break
3.15-4.30pm – COMMUNITY: Automating Database Deployments using Azure DevOps
4.30-4.45pm – break
4.45-6pm – AWS: 3-Step Guide to Modernize SQL Server to PostgreSQL
Thursday November 16
8.15-9.30am – KEYNOTE: The New Database Landscape – Revealing Shifts and Charting the Next Horizon
9.30-10.15am – break
10.15-11.30am – REDGATE: Redgate Product Launch: Transform your software delivery with Test Data Management
11.30am-12pm – break
12-1pm – LUNCHEON: Women in Technology: Got your Back: Finding Allies at Work
1-1.30pm – break
1.30-2.45pm – INTEL: How Microsoft & Intel Power Your SQL Server environments
2.45-3.15pm – break
3.15-4.30pm – COMMUNITY: Easy Enterprise Integration with Azure PaaS Services
4.30-4.45pm – break
4.45-6pm – COMMUNITY: Accelerated Database Recovery – A Deep Dive Behind the Magic
Friday November 17
8.15-9.30am – KEYNOTE: AIOps and ChatGPT: Prepare to Ride the Next Wave
9.30-10.15am – break
10.15-11.30am – REDGATE: Cloud Migration: The Good, the Bad, and How to Plan for Success
11.30am-1.30pm – break
1.30-2.45pm – MICROSOFT: What You Should Know About Always On Availability Groups
2.45-3.15pm – break
3:15-4:30pm – MICROSOFT: Everything You Need to Know About Data Virtualization in SQL
Lots of amazing sessions to watch.
I am very excited to share that I have been selected to speak at Data Weekender 6.5 on Saturday 4 November 2023.
The session I will be presenting is the Business Benefits of Good Governance at 9.15. See the schedule here.
Data Weekender is a Pop-up and Online Microsoft Data Conference. #DataWeekender is organised by a group of European-based Microsoft data platform professionals. The organisers are all regular contributors to the Microsoft Data Community either via speaking, blogging, hosting conferences, and running local meetup groups. Set up during the Covid-19 pandemic this was created to provide a simple way for people to continue presenting and hopefully for people to attend and learn. Since then the conference has grown from strength to strength.
