v1.17.0

Postgres Update

This version contains the first rewrite of the database modules to allow for running multiple connection configurations in parallel instead of relying on a global configuration. This refactor will eventually make it to the other database modules in future versions.

• The lapis.db.postgres module is now a proxy object for the "default" connection configuration
• Add configure function to lapis.db.postgres to define a new connection configuration. (Returns a compatible db object)
• Add logger field to db object. The db object will use that logger for any logging actions (queries, connections). Defaults to the module lapis.logger
• Add db_connection logger function to lapis.logger. The default implementation is empty and is designed to be overridden.
• Postgres connection now emits logging.db_connection(db, pgmoon, success, err) so you can add metrics on connections as you see fit
• Removed the pgmoon_conn performance fields, they were unreliable given that multiple connections could be created in a single request
• Add explicit disconnect and connect functions to the db module. The connection configuration can automatically reconnect if the connection has been disconnected.
• Note: The entire lapis.db.postgres module has mostly been rewritten, if you were relying on undocumented exports for custom behavior you should review your code before deploying

Other additions

• Added timer_at to lapis.nginx, a safe wrapper around ngx.timer.at that runs callbacks under pcall, bubbles errors, and guarantees after_dispatch is called to clean up any database connections
• After dispatch callbacks are removed after running to prevent double running if manually called
• lapis.http defaults to socket.http unless the cqueues is explicitly running (previously it would try to load lua-http)
• params_shape accepts label = false, letting you hide the automatic field: prefix when you want bare error strings or custom labels.
• Add support for dot syntax in default Nginx config template syntax to access fields of tables, eg. ${{postgres.host}}

Fixes

• Fix a bug where a nested preload could try to load relations on a basic value type that was returned by a relation

CLI

• Top level lapis command supports a --dir flag (requires luafilesystem), so you can run lapis from outside the project root
• Add lapis mcp shortcut for the lapis-mcp package
• When using lapis generate migration the name of the generated migration is now printed to stdout

v1.16.0

Install

luarocks install lapis

Additions

• lapis.validate.types -- Add types.params_map validation type, the params compatible variant of types.map_of

Changes

• model:update will now only assign the update object to the model instance if the update completes successfully
• model:update support the returns option to control the RETURNING clause of the generated query
• model:update when timestamps are enabled, the generated updated_at value is assigned to the model instance

Fixes

• lapis.validate.types -- Fix bug where types.params_shape would not return the state object
• model:update will avoid storing db.raw values on passed as update object to the model instance if the update does not complmete successfully

v1.15.0

Install

luarocks install lapis

Additions

• Model:include_in can now use computed keys to dynamically calculate a foreign key value by applying a function to each passed in object to load. This can be done by specifying a function instead of a field name when defining the column mapping table
• Relations can use compured keys where appropriate by passing a function instead of a field name when defining the column mapping table
• lapis.validate.types -- add types.params_array for validating an array of objects with a common shape
• lapis.validate.types -- add types.flatten_errors for error output compatibility with tableshape
• lapis.validate.types -- types.params_shape can now accept numerical names for fields for validating array like objects with a fixed number of entries
• lapis generate -- Rockspec generator can now specify --moonscript and --cqueues to automatically append dependencies
• lapis migrate -- Add the --dry-run flag to to run all pending migrations in a transaction that is never commited. (Note: in some databases, there are queries that can not be rolled back)

Misc

• Various updates to documentation
• Fix error message for types.truncated_text

v1.14.0

Install

luarocks install lapis

Additions

• Add --rockspec command to lapis new
• Add lapis generate migration command to either create blank migration file or append a new migration at the end of the existing migration file. See lapis generate migration --help for more information
  • Can append to Lua and MoonScript migrations files. It may not be able to edit all types of migration files, ensure that you return a simple table.
  • Support for appending either timestamp named migration or incrementing counter named migration
• Add lapis generate rockspec command to create a new rockspec for managing app dependencies. See lapis generate rockspec --help for more information
  • It will automatically detect the app name fro the current directory, if not specified with --app-name
  • It will automatically detect the git source URL from the current directory if a git repository has been created and has a remote
  • Additional initial dependencies can be included with flags like --postgres, --sqlite, --moonscript, etc.

Fixes

• Fix code reloading for cqueues when changes to package.path, package.cpath, package.searchers, and package.loaders is involved
  • These values will be reset to prevent infinitely growing module search path when reloading code
• Fix bug with lapis simulate command where header output wouldn't be normalized correctly
• Fix bug where lapis.validate.types params_shape was not correctly passing initial state object through the validation

Changes

• Update custom sub-command runner to support running commands with argparse specification

  • Update built in support for lapis annotate and lapis sytemd to work with this new system. It is necessary to update both of those packages if you wish to use them with this version of Lapis

• Rewrite all template files that lapis new creates as generator modules. All of these files can be independently created using the lapis generate command if necessary, eg. lapis generate app

  • lapis new will now internally call lapis generate to create the necessary files
  • lapis generate {NAME} --help can now be used for every template generator to view configuration options
  • List of updated generators: (these can be called with lapis generate {NAME})
    • app – For creating initial lapis application file in (app.lua or app.moon)
    • config – For creating config.lua or config.moon
    • models – For creating autoloader modules.lua or modules.moon module
    • gitignore – For creating initial .gitignore when using lapis new --git
    • tupfile – For creating initial Tupfile and Tuprules.tup when using lapis new --tup
    • nginx.config – For creating initial nginx.conf
    • nginx.mime_types – For creating mime.types nginx configuration include

• The default generated config for cqueues on MoonScript now longer includes fields for code_cache and num_workers, and instead now matches the Lua version of the config.

• lapis.db db.clause supports a prefix option to optionally append something to the front of the encoded clause if it contains any fields.

• lapis.db.model Generic preload can take the a callback function with the name of a relation to execute a callback on the loaded objects. Eg. `preload(users, { profile = function(profiles) ... end })

• Model:include_in add skip_included option to do nothing on objects that already have the related field loaded

• Model:include_in add loaded_results_callback option to provide a function callback to be called with with the list of objects that were fetched from the query

• Model:include_in when loading objects by a composite key, duplicate composite values will be stripped to reduce the size of the query generated (Singular keys already functioned this way)

• lapis.db.model.relations mark_loaded_relations can now take a third argument of a value to set for the relation (defaults to true)

• lapis.util singularize function has been improved slightly with more cases (eg. vertex, child, index, status) and will work with all capital words. Note that this function will never be comprehensive, only suitable for calculating a quick default in scenarios where names aren't explicitly specified

• lapis.validate.types Add multi_params for joining two params_shape objects together

Internal Changes

• Rewrite generic preload
• Add much more comprehensive test suite for generic preload

Full Changelog: v1.13.1...v1.14.0

v1.13.0

Install

luarocks install lapis

Additions

• Add the lapis simulate command to simulate a request to the application
  without starting a server. This is useful for testing and debugging. See
  lapis simulate --help for details about usage.
• Add support for SQLite (See lapis.db.sqlite, lapis.db.sqlite.model,
  lapis.db.sqlite.schema`)
• Add request:get_request() method to get a reference to the request object.
  (Note this just returns self, but it is useful in cases where the request
  object is being proxied through the helper chain of a widget)

Changes

• The default error page will now render a JSON response if the accept
  request header is set to application/json
• content_for now stores content blocks directly on the request object
  (prefixed with _content_for). This may solve a bug where widget helper
  chain could have cached an outdated copy of the content_for state.
• content_for will now throw an error if attempted to be used in a context
  without a request object
• Layout content rendering has been simplified to avoid additional closure and
  function scope created
• The resolved layout is now written to request.options (aka self.options)
  so that the view can know if a layout will be rendered within the current
  request. Note: actions that skip layout by default (eg. json, redirect_to)
  will write false to options.layout.
• Internal refactors to all of the database modules, removal of some
  undocumented functions (removed set_backend, set_logger, get_logger, init_logger)
• Updated generated SQL for MySQL in some places to output keywords in capitals
• lapis.db.pagination Paginators now have a simple fallback for clause
  generation when the database module lacks a clause parser. (Will work for
  simple queries without aggregates instead of throwing an error when calling
  total_items and has_items)
• lapis.spec Remove undocumented test helpers: use_db_connection, assert_no_queries.
• lapis.spec use_test_env and use_test_server no longer interact with
  database connection in setup and teardown.
• lapis.spec.request The mocked ngx object now covers more fields to make
  it suitable for more kinds of requests

Full Changelog: v1.12.0...v1.13.0

v1.12.0

Install

luarocks install lapis

Additions

• Add the lapis.validate.types module with tableshape compatible types for parameter validation: params_shape, assert_error, cleaned_text, valid_text, trimmed_text, truncated_text, limited_text, empty, file_upload, db_id, db_enum
• lapis.validate Add the with_params helper function for wrapping action function with parameter validation
• lapis.util.utf8 Add string_length function for counting the number of characters in a string
• The lapis new command will always write a config.lua/config.moon file. The default configuration's nginx specific variables will be phased out a future update
• The lapis generate sub commands now can generate both Lua and MoonScript files. Project type detection has been added, in addition to flags to directly specify the file type

Changes

• The lapis command line tool's argument parsing has been rewritten using argparse. New help commands are available for every command, with full argument documentation.
• The lapis generate subcommands now use argparse and have full command line documentation available
• The lapis new command will now fail if the requested server type is not available. The --force option can be used to bypass the error
• The default nginx.conf file now include an init_by_lua block as an example to suppress global variable warning
• Command line third-party sub commands now are executed using the _ command. See lapis help _
• lapis.application yield_error now has a default error message if one isn't provided
• lapis.validate assert_valid can take a tableshape object as the validation object. Will return transformed value and state on success
• lapis.db encode_clause will now generate query with capital NOT when using false value

Misc

• Internal reorganization for command line commands and path library
• Revised documentation for input validation

Full Changelog: v1.11.1...v1.12.0

v1.11.1

Fixes

• db.clause supports having a db.list object as the key of a field to represent composite tuple (matches behavior of bare table passed to db.encode_clause)
  • fixes regression where where: clause could not be combined with composite key in model:include_in due to the refactored clause generation implemented in 1.10.0

Changes

• preload will only try to process sub-relations if the value of the field is a string or table

Full Changelog: v1.11.0...v1.11.1

v1.11.0

Install

luarocks install lapis

Highlights

• Substantially improved performance for url_for generation
• Introduce db.clause
• Lua is now first class. Etlua improvements. Application class APIs match Lua & MoonScript. Documentation and APIs show Lua by default
• Homepage and documentation have been substantially updated

Additions

• lapis.db Add the db.clause constructor for safely building SQL fragments including clauses and conditionals
• All db related methods can now take db.clause objects when appropriate for configuring filtering (eg. Model:find, relation where: clause, etc.)
• lapis.db The lapis.db.encode_clause function can be used to convert a db.clause object a fragment of SQL code
• lapis.application Add Application:extend() method to create a sub-class of Application when MoonScript is not available
• lapis.application All route related methods are now consistently available for both application instances and classes. This includes include, match, extend, find_action, before_filter, get, post, put, delete
• lapis.session The get_session function can now take a string object as first argument, as an alternative to the request object, to decode a session from a cookie's string value
• lapis.html Add Widget:extend() method for creating sublcass of the Widget class when MoonScript is not available
• lapis.html The classnames function will now recurisvely evaluate any tables in the argument
• lapis.html Add is_mixins_class function to determine if a class is a dynamically generated mixin class created by Widget:include
• lapis.etlua The element function has been added to the template scope to allow rendering HTML elements programmatically (similar to the HTML builder syntax)
• lapis.etlua self has been added to the template scope to allow accessing the instance of the EtluaWidget
• The lapis migrate command now supports a --transaction flag, can be set to global to apply a transaction across all migrations to be run, or individual to apply a transaction to each migration run.
• lapis.flow Add Flow:extend() method for sublcassing the Flow class

Changes

• lapis.db Internally, queries now use db.clause to generate SQL conditional statements . This means that order of certain fields may now be different when using where clauses.
• lapis.db It is no longer possible to override fields configured by a relation when specifying a where: option. This also applies to paginators generated for relations
• lapis.db.model Relations can now be specified with a direct reference to a Model class, or a function that should resolve to a Model class (existing support for relation name has not changed)
• lapis.application Inheritance of routes is now more well defined, allowing for route names and paths to be overidden by subclasses or instances during the creation of the router. If you aren't using application inheritance then this will not affect you.
• lapis.application Lazy action loading is now supported for actions generated by include and actions built by the HTTP verb match helpers (get, post, etc.). Previously, if provided an action name (or true), it would load the action module immediately. Now all named actions are consistently loaded on first request regardless of where they are used
• lapis.application The find_action method now searches up inheritance hierarchy, and can be used on both app classes and instances
• lapis.application Trying to call enable or match directly on the lapis.Application class reference will now throw an error to help prevent accidentally mutating the global object
• lapis.html The Widget:render() method will now return nothing instead of nil
• lapis.router Router URL creation (aka url_for) has been rewritten to be substantially faster. Previously routes were re-parsed on calls to url_for but will not generate from a cached intermediate form that will allow the URL to be generated with little overhead.
• lapis.cqueues Add error capturing around the app boot process to provide better error message when attempting to load a faulty app, and prevent infinite loop processing bug from lua-http
• The environment variable LAPIS_FORCE_LOGGING can not be set to 0 to force logging off
• The application field is no longer present on the lapis.init module. This was never documented. Use require("lapis.application") instead.
• Many error messages have been rewritten to be prefixed with the module or method they originate from

Internal Changes

These changes should have no effect on the end user implementing an app, but
they are documented here in case you were depending on the undocumented
structure of lapis

• lapis.application The internal structure used by the HTTP verb match helpers has been changed (get, post, etc.)
• lapis.router Parsed routes are now stored, and the arguments for fill_path have changed
• lapis.application The way routes are internally managed and iterated has been rewritten to provide unified interface based on metatable inheritance. The lapis.application.route_group module has been added to work with this interface.

Misc

• Substantial updates to documentation, including rewrites for clarify and documenting fields that were previously not documented.
• Changes to the homepage to prioritize Lua syntax over MoonScript to reduce confusion about what & who the framework is for

Full Changelog: v1.10.0...v1.11.0

v1.10.0

Install

luarocks install lapis

Additions

• Model:include_in supports a new { load = false } option to disable the conversion of query results to model instances
• You can now specify { order = false } when creating a paginator from a model relation to strip the default order that may have been specified in the relation definition
• lapis.util.utf8: Add a UTF8 aware trim LPeg pattern, called trim) (Note: the utility function trim located lapis.util still only operates on ascii whitespace)

Postgres

• model:update method can take a where clause to apply a conditional update
• The return value of model:update is now well defined, and works similar to model:delete. Will return boolean true/false depending on if the update was able complete, and the resulting object from the update query. (Warning: Previously model:update would return the result object regardless of success as the first return value, but this functionality was undocumented.)
• db.insert can now take options as a table, and supports receiving returning columns as a option
• Add the { on_conflict = "do_nothing" } option to db.insert to not throw an error if insertion is canceled due to a unique key constraint conflict (using the ON CONFLICT DO NOTHING query syntax introduced in Postgres 9.5)

MySQL

• model:update will return true if the number of affected rows is greater than 0, followed by the result object

Changes

• lapis.html: All element helper methods (eg. div, b, span, etc.) now return nil instead of the previously undocumented behavior of returning the buffer object. This is to prevent accidentally leaking data when writing malformed syntax such as div div! (instead of the correct (div -> div!)
• lapis.util.utf8: whitespace pattern is aware of invalid use of direction markers
• lapis.db.postgres.model: On model creation, if a returning * clause is provided then unnecessary extra returning fields are not included in the query
• lapis.db.postgres: Nginx environment detection is more accurate to allow pgmoon connections to be created without error in more stages of the Nginx worker/request lifecycle
• lapis.db.postgres: OpenResty specific performance metrics about socket reuse are only written when an nginx socket is in use
• lapis.db.postgres: Add connection specific performance metric for when any socket type other than nginx is used

Bugs

• lapis.validate: Fixed bug where the input options table passed into validate would get mutated, removing the optional field

Misc

• Assorted updates to docs
• Expanded test suite, refactors to tests
• Documentation typo fixes by @tommy-mor in #734 #746

New Contributors

• @tommy-mor made their first contribution in #734

Full Changelog: v1.9.0...v1.10.0

v1.9.0

Changes

• Widget\include is now implemented completely different: A dynamic mixins class is generated and inserted into the class hierarchy of the widget when include is first called. All fields and methods from any included classes are copied into this mixin class.
  • It's now possible to override a method that is provided by an included class. (The current class has higher precedence than the mixins class)
  • super can be used to call the method provided by the included by the class when overriding
  • Some undocumented functionality regarding method merging was removed
  • Only one mixin class will be inserted, and all included classes will copy their fields into that class. The most recently included classes will override anything that has the same time. (Note: there is no hierarchy for multiple includes, so you can't use super to call through multiple included classes)
  • Methods from the class hierarchy of the included class are also copied into the mixin class. (Note: methods are copied, and the hierarchy is flattened, meaning methods that depend on super may not work as intended)
  • Note: The "dynamic mixin class" approach is also used for models when relation methods are generated
• lapis.spec.request.stub_request no longer creates an anonymous app subclass, instead it will override the dispatch method on an instance of the class before stubbing the request
• lapis.spec.request.mock_action will no longer push the test environment on every call, but instead will use the current environment. (Note: v1.8.2 added autodetection of busted to ensure specs are always running the test environment, so it's not necessary to manually set the environment)

Additions

• Add support for on_invalid_method option to respond_to
• fetch relation can be set to true to autogenerate a get_ method based on the provided preload function
• Added support for optional reloations with db.preload. Relation names prefixed with ? will be ignored if the object being preloaded does not contain a relation with that name.
  • For example, trying to preload the relation hello when the model does not have hello will result in an error, but preloading ?hello will do nothing.
  • Optional ? relations can have nested preloads, if the relation is not found then none of the nested preloads are loaded. This works best when combined with polymorphic relations where the objects may not all share the same relation interface.

Fixes

• Lazy loaded actions work correctly when merged from sub applications
• Before filters work correctly with lazy loaded actions
• Lazy loaded actions work with Lua method responders

Misc

• Default error page has encoding set to UTF-8
• Simplify implmentation of preload_relations, deprecate it
• Optimize concatenations db.encode_values
• Improve error messages for some functions
• db.encode_assigns, db.encode_clause, db.encode_values throws error on empty table instead of generating invalid SQL from empty table
• sorted_pairs spec helper
• Additional specs (respond_to, widget including, lazy loaded actions)