mirror of
https://github.com/microsoft/fabric-cicd.git
synced 2026-11-04 12:02:01 +00:00
main
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  
|
e52d7383a3 |
Fixes #567 - Add $items notation support to key_value_replace (#643)
## Description
The `$items` notation (e.g., `$items.Lakehouse.Sample_LH.$id`) was only
available in `find_replace` parameters. This PR extends support to
`key_value_replace`, enabling dynamic item references in JSON/YAML files
using JSONPath expressions.
### Changes
- **Modified `replace_key_value()` in `_utils.py`**: Process string
values through `extract_replace_value()` to handle `$items` notation,
reusing existing logic from `find_replace`
- **Added test coverage**: Two new tests verify `$items` notation works
correctly with string values while preserving existing behavior for
other types (boolean, integer, etc.)
- **Updated documentation**: Added examples using realistic DataPipeline
JSON structures with JSONPath targeting `notebookId` and `workspaceId`
properties in actual `pipeline-content.json` files, clarified that
dynamic replacement applies to both parameter types
- **Updated sample parameter file**: Changed example to use DataPipeline
item type with JSONPath targeting the "Run Hello World" pipeline
structure
### Example Usage
```yaml
key_value_replace:
- find_key: $.properties.activities[?(@.name=="Run Notebook")].typeProperties.notebookId
replace_value:
PPE: "$items.Notebook.Hello World.$id"
PROD: "$items.Notebook.Hello World.$id"
item_type: "DataPipeline"
- find_key: $.properties.activities[?(@.name=="Run Notebook")].typeProperties.workspaceId
replace_value:
PPE: "$workspace.$id"
PROD: "$workspace.$id"
item_type: "DataPipeline"
```
Supports the same attributes as `find_replace`: `$id`, `$sqlendpoint`,
`$queryserviceuri`, and all workspace variables.
## Linked Issue (REQUIRED)
Fixes #520
<!-- START COPILOT CODING AGENT SUFFIX -->
<details>
<summary>Original prompt</summary>
>
> ----
>
> *This section details on the original issue you should resolve*
>
> <issue_title>Allow for using $items notation also in
key_value_replace</issue_title>
> <issue_description>### What is the feature?
>
> The $items notation is really nice but currently it only works in
find_replace parameter definitions. It would be highly valuable if this
worked in the key_value_replace also.
>
> ### Additional context
>
> _No response_</issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
> <comments>
> <comment_new><author>@shirasassoon</author><body>
> @celindho Thanks for providing this feedback. Happy to look into this.
>
> I would like to get your thoughts, would you expect the same attribute
options (id, sqlendpoint, etc.) or could you see a need for any
additional attributes that would be required in the key_value_replace
scenario?</body></comment_new>
> </comments>
>
</details>
- Fixes microsoft/fabric-cicd#567
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/microsoft/fabric-cicd/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: shirasassoon <66449905+shirasassoon@users.noreply.github.com>
Co-authored-by: Shira Sassoon <shirasassoon@microsoft.com>
|
||
 Per Christian Kofstad
|
843fff62ee |
#588 Add support for UserDataFunction (#603)
# Fixes #588 - Add support for UserDataFunction item type ## 📋 Summary This PR adds comprehensive support for **UserDataFunction** item type to the fabric-cicd library, enabling CI/CD workflows for Microsoft Fabric User Data Functions. This resolves the error `Invalid or unsupported item type: 'UserDataFunction'` reported in issue #588. ## 🚀 What's Changed ### Core Library Implementation - **Added UserDataFunction to accepted item types** in `constants.py` - **Created dedicated publish module** `_userdatafunction.py` following established patterns - **Integrated publishing logic** into main workflows in `publish.py` - **Updated module exports** in `_items/__init__.py` ### Documentation & Examples - **Added comprehensive documentation section** in `item_types.md` with parameterization guidance - **Created complete sample structure** demonstrating proper UserDataFunction folder layout - **Included example function code** with HTTP triggers and business logic ### Files Modified ``` src/fabric_cicd/constants.py - Added UserDataFunction to ACCEPTED_ITEM_TYPES src/fabric_cicd/_items/_userdatafunction.py - New publish module (23 lines) src/fabric_cicd/_items/__init__.py - Added import/export src/fabric_cicd/publish.py - Integrated into publish/unpublish workflows docs/how_to/item_types.md - Added documentation section sample/workspace/SampleUserDataFunction.UserDataFunction/ - Complete sample implementation ``` ## 💡 Implementation Details ### Following Established Patterns This implementation closely follows the pattern established by similar item types like `DataAgent` (PR #580): - Simple `_publish_item` call without special exclusions - Proper alphabetical ordering in publish/unpublish sequences - Standard error handling and logging patterns - Consistent module structure and naming conventions ### Sample Structure Included Created a complete sample UserDataFunction demonstrating the expected folder structure: ``` SampleUserDataFunction.UserDataFunction/ ├── .platform # System metadata (version, logicalId, type, displayName) ├── definitions.json # User Data Function definitions (connections, libraries, environment) ├── function-app.py # Your functions code with HTTP triggers └── resources/ └── functions.json # Function metadata (auto-managed by Fabric) ``` ### Key Features Supported ✅ **Git Integration** - Full source control support ✅ **Deployment Pipelines** - Automated deployment across environments ✅ **Parameterization** - Connection and library references via `find_replace` ✅ **Library Management** - Automatic handling of Python dependencies ✅ **Function Code** - Version control for business logic ## 📖 Documentation Added documentation section including: - **Parameterization guidance** for connections and libraries - **Initial deployment requirements** and behavior - **Function code deployment** details - **Metadata management** information - **HTTP trigger support** and business logic capabilities ## 🧪 Testing - ✅ Library imports successfully with new changes - ✅ Constants properly include UserDataFunction - ✅ New publish function imports correctly - ✅ Code passes ruff formatting and linting - ✅ No breaking changes to existing functionality - ✅ Manually tested and deployed UserDataFunction from local computer to new workspace ## 📚 References - **Issue**: #588 - Add support for UserDataFunction - **Microsoft Documentation**: [User Data Functions Git Integration](https://learn.microsoft.com/en-us/fabric/data-engineering/user-data-functions/git-and-deployment-pipelines) |