RoadmapFrends

Frends Official Task Development Guidelines

What requirements there are to publish an official Frends Task.

While everyone can develop Custom Tasks for their own use, in order to publish them as official Frends or Partner Tasks, the following guidelines should be followed. These are the guidelines we follow internally, and expect all published Tasks to follow as well.

Note regarding creating Partner Tasks: Partner Tasks cannot use the namespace "Frends." as that is reserved for Official Frends-developed Tasks. Partner Tasks should instead use their own partner prefix, e.g. "Company".

Technical Setup

Use Frends task template for creating new tasks. See GitHub - FrendsPlatform/FrendsTaskTemplate: Template for creating Frends Tasks.

Template will automatically handle the following:

  • Target .NET 8

  • Task project namespace should be Frends.SYSTEM.ACTION

  • Task class name should be SYSTEM

  • Task method name should be ACTION

    • Example: namespace = Frends.Salesforce.Query, class name = Salesforce (in namespace Frends.Salesforce.Query) and method name = Query

  • Unit test project should be present

Unit testing

  • We require at least 80% unit test coverage

  • The code should use dotenv for managing secrets

  • If Docker is used, make sure to mention how to set it up

    • For Docker use docker-compose.yml

    • For credentials for 3rd party software we use 1Password for storing credentials

  • If no Docker container exists for the system, try to mock the services inside the unit test project

    • If mocking is unrealistic and we need to use the actual system, mention in readme how to set up e.g. a test account for that system.

Repository structure

Place GitHub Actions inside .github/workflows, because this is the only place they can be in.

Each Task should have its own subfolder. The subfolder should contain Task-specific solution, that in general contains the task project and test project.

Each Task should have a separate solution without code sharing between them. Tasks should be atomic with as little external dependencies as possible.

This is because using the same external code in multiple Tasks means that any changes to the shared code need to trigger multiple Task releases, and this will be definitely very hard to track and remember.

Thus we opted for separate Task solutions, so that it is clear that Tasks are individual and separate entities.

References in to NuGets

If some NuGet package is used in several projects, its version should be the same to avoid problems with conflicting dependency versions.

Licenses

Tasks by default always use MIT license.

For dependencies we must always check their license and make sure that it is a permissive license. The only permitted permissive licenses for us are:

  • MIT

  • Apache 2.0

  • BSD

Everything else, including GPL, AGPL, LGPL, is forbidden and must not be used.

We must always make sure to respect the license terms, such as attribution (e.g. in the README.md).

Task versioning

We use the standard versioning conventions with Tasks with three numbers, e.g. 1.2.4.

  • Bump the first number if you are making breaking changes to the Task. A non-exhaustive list of breaking changes:

    • Moving Task Parameter from one tab to another (e.g. from Input to Options)

    • Renaming a Task Parameter (like fixing a typo “ConnectionStrung” → “ConnectionString”)

    • Removing or renaming a tab

    • Adding a new Task Parameter that does not have a default value, which would replicate the old Task functionality (== altering Task behaviour)

  • Bump the second number for non-breaking changes. Some examples:

    • Fixing/improving Task documentation

    • Adding new Task Parameters that with default values will NOT alter Task behaviour

  • Last number is reserved for developers. Frends does not allow importing same Task versions more than once, so if you are testing a Task with Frends you will need to bump the last version number.

When creating new versions of Tasks it is preferable to make the edits non-breaking. For example, if adding a new Task Parameter, try to see if it can have a default value that will not break the current Task functionality. However, also consider if this is a sane default.

Builds and releases

Builds must never have any warnings in them.

Sometimes there are cases when build generated irrelevant / unuseful warnings - in these cases you are allowed to mute those warnings with #pragma. However, you should always explain the #pragma in a comment - why it is here and why it is ok to mute the warnings. Pragma blocks without explanation are not allowed.

The Task template will set up your Task to use the standard GitHub Actions workflows by default and those will be in the .github folder in your Task repository.

We have just a few workflows that make building and publishing work and they can be found in GitHub - FrendsPlatform/FrendsTasks . Every Task repository will use those workflows. There are some exceptional cases, when our workflows don’t fit. In those cases, talk to the team at Frends for instructions.

Docker

We use Docker for simulating target systems, and whenever possible, use Docker to spin up a test instance of the software that you are making the Task for. For example, most databases have ready images.

Use docker compose files to make it easy for others to spin up the same setup. We also have a parameter in GitHub workflows to allow spinning up the containers when running tests.

Misc

  • FrendsTaskMetadata.json must exist in Task project folder (handled by Task template)

  • Bagdes must have correct and have proper values (usually handled by Task template)

    • Main build

    • Nuget

    • License

    • Code coverage

  • README.md in repository's root folder must have link to Task README.md

  • Task parameters and result must not include any 3rd party classes.

    • For example, you cannot use System.Security.Claims.ClaimsPrincipal in your result object.

    • Instead you should create a new class which will contain all the necessary information.

  • Here is a basic example of what properties should be in the task project file:

Target .NET 8

Target .NET 8 in all new Task development. In rare cases we might need to use .NET 6. Every case of using .NET 6 has to have a good reason for it and needs approval.

For .NET 8 tasks our GitHub workflows have a parameter called dotnet_version, which you can specify to make sure that workflows can compile your task successfully. Example:

... jobs: build: uses: FrendsPlatform/FrendsTasks/.github/workflows/linux_build_test.yml@main with: workdir: Frends.MQTT.Receive dotnet_version: 8.0.x prebuild_command: ... ... ...

Documentation

Each Task package should include the following documentation.

README.md

This is mostly basic information for developers. It should contain the usual text (generated by the Task template), including badges and some description text.

In addition to standard text, use this file to indicate if something special needs to be set up by the developer when starting work. For example:

  • Anything around certificates or getting the unit tests to work properly

  • How to get credentials for systems, if this way exists

  • Any additional setup, such as installing other software or obtaining licensed and purchaseable library

Readme should not include any manually written Task Parameter descriptions or such. This file is only for the technical setup and should not act as enduser documentation.

CHANGELOG.md

Changelog must be always up to date when making edits to a Task. Every functional change should be reflected here.

Changelog should be added to nugetpackage (same as metadata file). Otherwise, changelog will be fetched from Task repository, but only if repository follows standard Task structure.

Changelog file uses the Keep a Changelog format.

However, changelog should not have notes on internal work, if it does not affect functionality of the Task. For example, instead of “refactored some code in MyBestClass.cs”, write why you did it. It can be either functional stuff, or if there were no functional changes, then a brief “Code cleanup” will suffice.

Breaking changes should always describe how to upgrade to the new Task version. For example:

Task Documentation

Task XML summary must link to Task readme in markdown format [Documentation](link).

All public members must be documented.

All parameters and result properties must be documented and have examples (in <example> blocks).

For showing a data structure e.g. in task <result> documentation use the following format:

  • object { bool Success, string Error }

  • object { bool Success, object[] Messages { string MessageId, string MessageText } }

  • If the object is big - use newlines to make things readable.

For extended Task documentation do not use <summary> (it will upset Frends UI), but use a custom tag - <frendsdocs>. This will be put directly under summary in our documentation website. You can (and probably should) use Markdown in both summary and frendsdocs.

Do not use crefs and similar tags inside of XML documentation texts.

Task Conventions

Tasks

  • PascalCase is the default naming convention for tasks. This includes abbreviations in task names as well.

  • Task names should be short and punctual stating the purpose of the task.

  • Task name should include the entity type, e.g. UploadObject, ListFiles, ReadBlob.

Parameter Names

  • PascalCase is the default naming convention for parameter names

  • In general, only the most common abbreviations should be used

    • For example file name formats, API and URL

  • All abbreviations should follow the PascalCase naming

    • For example, CSV = Csv or URL = Url

  • Parameter names should be descriptive but brief

    • For example, use InputFilePath or OutputFilePath instead of FilePath

  • Whole words should be used instead of abbreviations of words

    • For example, use “Arguments” instead of “Args”

  • Avoid repeating the name of the system in the names of parameters

Parameter descriptions

  • Description should include more thoroughly the properties of the parameter, i.e. what it does.

  • Use case examples are encouraged.

  • Links to external websites can be included, if additional information can be found on the website.

Task tabs (input, options etc.)

  • Each Task should have tabs for at least Input. More tabs can exist if necessary. Use the appropriate tab for the parameters (see below).

  • Input

    • The Input tab should always be the first tab.

    • It should contain the parameters that are essential for the Task to function.

  • Connection

    • If multiple fields for credentials exist and Input becomes excessively large, a Connection tab should be used.

    • For API URLs, access tokens, client IDs & secrets, connection strings etc.

  • Options

    • Should contain additional and/or optional parameters for configuring the Task. Note that connection-related parameters belong in the Connection tab.

    • For Task behaviour control every Task should have ThrowErrorOnFailure

      • this should be TRUE by default

    • Also have parameter ErrorMessageOnFailure

      • If throw exception, then we wrap the Task exception with a new one with ErrorMessageOnFailure text (or fallback on default error message)

      • If not throw exception, then result ErrorMessage should contain ErrorMessageOnFailure and task own error message.

  • Source and Destination

    • Source and Destination tabs must be moved to Connection tab!

    • These tabs should be used if there is a lot of parameters for each. This could be the case with e.g. download and upload Tasks.

Task result

  • For consistency, every result should always return a Boolean value “Success”.

  • If a Task returns some data (database rows or other weakly defined data) on success, it should be called simply “Data”.

    • If we know the result items well in advance, then the result properties should be called semantically, by their meaning

    • Examples:

      • Database retuned rows should be Data

      • Snowflake should return Data

      • ListFiles result should be e.g. FilePaths

      • FileExists result should be e.g. FileExists

  • Other parameter names for results vary depending on the type of the source system. See Parameters-section for Task-specific values.

  • Errors

    • In case of errors the result object should contain Error property, of the following format:

      • { Message: “error text, must always be present”, AdditionalInfo: [ any object type, depending on task ] }

    • The AdditionalInfo can contain any object type and does not HAVE to be dynamic. Only use dynamic when we have an unknown structure there.

    • The AdditionalInfo should always be describing the problem as accurately as possible. E.g. in case of multiple file copy we would want to include the file names that failed to copy.

      • Even if the file copy failed on some files, then we should still try to include the actual exceptions in the output, so it can be debugged by developers.

    • If there is not possibility to generate meaningful error message(s), or if the “global” try-catch fails, then AdditionalInfo should contain the exception thrown.

  • If result is containing some property that has undefined content, the it must be dynamic and actual type JToken

  • For Tasks that attempt to do multiple operations (like upload multiple files one by one, update several Event hub partitions etc.)

    • If one of the operations fails - still continue and try to do other operations (if that makes logical sense to continue)

    • Set the result Success to false and include all the failed operations in AdditionalInfo

    • If throw on failure is set to true, then continue to do the operations and then in the end, if some of operations failed - throw an exception

Quick and simple example of the result object:

Task type-specific guidelines

The abovementioned general guidelines apply to all Tasks. However, some types of Tasks have special properties that should be noted.

Databases

The name of the input field for a database query should always be “Query”.

Similarly, database requests should always return “Data” and “RecordsAffected” (however, records affected should be present only if the database provides this data itself for e.g. updates/deletes)

The base structure of parameters and their order in a basic database query Input should be, when feasible:

  • Connection string must be under Connection tab

  • Input tab:

    • Query

    • Parameters

  • Options tab

    • Other options like e.g. transaction isolation level etc

Tasks that utilize a REST API

The base structure of parameters and their order in a basic REST API task Input should be:

  • Method if applicable

  • Url / Domain

  • Api version if applicable

  • Message

  • Credentials → Connection tab

The base structure of the result should be:

  • Body

  • StatusCode

Converters

Consists of conversions from one file or data type to another, such as “ConvertJsonToXml”.

Converters are usually simple Tasks on the outside:

  • Input tab parameter should include the name of the input type, such as “Json”.

  • Result object should include the name of the output type, such as “Xml”.

File operations

Consists of file transfers from one system to another, for example local-local, local-SFTP, SFTP-SFTP, SFTP-local, local-Azure Data Lake etc.

The base structure of parameters and their order in a basic file operation Task the Input should be:

  • Input tab

    • Local filesystem information, like Source or Destination file paths

    • Remote filesystem information, target upload folder etc.

    • If file conflicts need to be handled, then the name of the property is ActionOnExistingFile and it goes to the Input tab.

  • Connection

    • Server info, like SFTP/Azure Data Lake connection info

  • InputFilePath

    • If only a single file is handled, the full path to the file should be used instead of separating the directory and the filename.

  • DestinationFilePath

    • If only a single file is handled, the full path to the file should be used instead of separating the directory and the filename.

  • Results:

    • Tasks that involve reading the files should return “Content”.

    • Write, move, rename, download and upload related tasks should return “FilePath(s)” or “Url” where applicable.

    • List and delete tasks should return “FileName(s)”.

Task resource disposal

In some cases resources cannot be unloaded during the Task execution. For those special cases the Task should clean up things when the Assembly Load Context is being unloaded. Here is an example of this:

PreviousSequence FlowNextPublishing Tasks to Frends Official Channels

Last updated

Was this helpful?